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Introduction to the Quality 
Improvement Center on Early 
Childhood 


In FY 2009, the Children’s Bureau funded the 
Center for the Study of Social Policy, in 
partnership with ZERO TO THREE: National 
Center for Infants, Toddlers, and Families, and 
the National Alliance of Children’s Trust and 
Prevention Funds, to create a National Quality 
Improvement Center on Early Childhood (Q1C- 
EC) focused on child maltreatment prevention. 
The purpose of the 5-year QIC-EC is to promote 
knowledge development, knowledge 

dissemination, and knowledge integration in this 
area. The logic model follows in Table 1. 

Knowledge development focuses on program 
and systems strategies that contribute to the 
prevention of child maltreatment and to the 
promotion of increased family strengths and 
optimal development among infants and young 
children (birth-5) who are at high-risk for abuse, 
neglect, and abandonment. 

Knowledge dissemination is supported by 
facilitating collaborative information- sharing and 
problem-solving via a national QIC-EC Learning 
Network, the Children’s Bureau Training and 
Technical Assistance network, and ongoing 
relationships with other stakeholders and partners. 

Knowledge integration is the culmination and 
desired impact of knowledge development and 
knowledge dissemination resulting in positive 
change for families and children and sustainable, 
systemic change at multiple levels of the child 


maltreatment prevention field. Integration 
happens as effective knowledge development and 
dissemination activities reinforce, support, and 
then translate new learning into practice and use. 

CONTENTS 

■ Affirmation of the Need for the 
QIC-EC, p. 3 

■ Designing the Delivery and 
Evaluation of an Intervention, 
p. 12 

■ The Overarching Research 
Question for the R&D Projects, 

p. 22 

■ Outcomes, Indicators, 
Measurement, and 
Instrumentation, p. 24 

■ Introduction to Evaluation Plans, 
p. 36 
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Table 1 : The QIC-EC Logic Model 


THE QIC-EC LOGIC MODEL 

Vision: Develop, disseminate, and foster the integration of a strong theoretical model about conceptualizing, researching, and evaluating child maltreatment prevention efforts in order to 
increase family strengths, promote optimal child development, and decrease the likelihood of maltreatment of children ages birth-5 who are at high risk for abuse, neglect, and abandonment and 
for whom there is no substantiated Child Protective Services report. 

Resources 

Activities 

Short-Term Outcomes (QIC-EC Years 2-5) 

Long-Term Outcomes (Impact) 

■ The Children’s Bureau 

■ The National Advisory 
Committee 

■ Partnering organizations (CSSP, 
ZTT, The Alliance) 

■ Evaluation Team (InSites) 

■ Project staff 

■ Subcontractors (NCCP, Chapin 
Hall) 

■ Commissioned papers 

■ Learning Network partners 

■ Website and other dissemination 
channels 

■ Funding for research and 
demonstration projects, 
dissertation projects, and other 
activities of the QIC 

■ Fund and provide support to 3-5 selected 
R&D projects that test and evaluate how and 
to what extent collaborative, innovative, 
evidence-based and evidence-informed 
program and systems strategies that increase 
protective factors and decrease risk factors 
result in optimal child development, increased 
family strengths, and decreased likelihood of 
child maltreatment within families of young 
children at high risk for maltreatment. 

■ Conduct a cross-site evaluation of the impact 
of funded R&D projects 

■ Establish a national information- sharing and 
communications network to engage a broad 
maltreatment prevention constituency, 
disseminate lessons learned from this initiative, 
and receive feedback 

■ Award up to four 2-year dissertation research 
stipends to advanced-level doctoral students 
conducting research on preventing the abuse 
and neglect of infants and young children and 
promoting child and family well-being 

■ Increased knowledge about evidence-based and 
evidence-informed program and systems strategies 
that contribute to the prevention of child 
maltreatment and to the promotion of increased 
family strengths and optimal child development 

■ Increased knowledge about maltreatment 
prevention research and evaluation methodology 

■ Wide-spread knowledge dissemination to 
practitioners, key policy makers, leaders, state and 
federal agencies, parents, and the general public 

■ Integration of knowledge that results in sustainable, 
systemic change at multiple levels of the child 
maltreatment prevention field 

■ Increased collaborations and collective problem- 
solving among prevention partners including 
members of the Children’s Bureau TTA Network, 
researchers, practitioners, and policy makers 

■ Increased available and accessible resource 
information on child maltreatment prevention, 
protective factors, promotion of family strengths, 
optimal child development, evaluation, diffusion of 
innovation, etc. 

■ Evidence-grounded recommendations about 
changes in practices, procedures, and policies 

For children and families: 

■ Strengthened family 
functioning 

■ Improved developmental 
outcomes 

■ Reduced likelihood of first 
occurrence of child abuse, 
neglect, abandonment, and 
entry into foster care system 

For systems: 

■ Strengthened and sustained 
collaborations across child and 
family service systems 

■ Proactive change within and 
across child and family service 
systems 

■ Proactive change in the general 
public’s attitude about child 
maltreatment 

■ Public policy that serves the 
well-being of all children 
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Affirmation of the Need for the QIC-EC 


Research and demonstration projects within the 
QIC-EC will test and rigorously evaluate program 
and systems strategies that contribute to the 
prevention of child maltreatment and to the 
promotion of increased family strengths and 
optimal development among infants and young 
children (birth— 5) who are at high risk for abuse, 
neglect, and abandonment — including those 
impacted by substance abuse or HIV/AIDS — and 
for whom there is no substantiated Child 
Protective Services report. The following sub- 
sections provide information and evidence that 
affirms the need for the QIC-EC and gives 
direction to the work of the R&D projects: (a) the 
need to focus on maltreatment prevention, (b) the 
need to use a social-ecological framework, (c) the 
need to focus on infancy and early childhood, (d) 
the need to focus on high-risk populations, and (e) 
the need for effective maltreatment prevention 
collaborations. 

The Need to Focus on 

Maltreatment 

Prevention 

Child maltreatment is not a new phenomenon; yet, 
more new knowledge is needed to understand and 
prevent its occurrence. 

A complete etiologic picture of child 
maltreatment is still emerging, and the 
myriad of risk and protective factors 
related to child maltreatment at all levels 
of the social ecology... remains 
unexplored.... Further, critical questions 
regarding the effectiveness of prevention 


programs still remain.. ..Rigorous research 
is needed to assess the effectiveness of 
prevention programs and to determine 
which among them merit widespread use 
(Centers for Disease Control and 
Prevention, 2009, p. 76). 

Historically, the majority of resources and efforts 
have been spent on responding to child 
maltreatment after it has occurred (tertiary 
prevention); thus, efforts addressing child 
maltreatment have typically targeted the child 
welfare system. While child protection services 
are vitally necessary, many agree they are not 
sufficient in the goal to prevent child abuse and 
neglect (Shaw & Kilburn, 2009). “Yet, despite the 
potential long-term benefits of preventing child 
abuse and neglect, only a small percentage of all 
resources specifically earmarked for child 
maltreatment in the United States is actually 
devoted to prevention” (Thomas, Leicht, Hughes, 
Madigan, & Dowell, 2003, p. 2). Thus, the need 
for a Quality Improvement Center that focuses on 
advancing knowledge and practice about effective 
primary prevention (before abuse and neglect 
occurs) and secondary prevention (focusing on 
those who are at high risk for maltreatment), is 
paramount. 

The Need to Use a 

Social-Ecological 

Framework 

The need to use a social-ecological conceptual 
framework grows out of the premise that human 
behavior does not occur in a vacuum. Children 
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develop within a network of family relationships, 
families exist within a community, and the 
community is surrounded by the larger society; 
these levels of the social ecology interact with and 
influence each other. Using a social-ecological 
framework expands the scope and reach of child 
maltreatment prevention efforts from a singular 
focus on individual factors to include 
relational/family factors, community factors, and 
societal/ systems factors and, thus, create a far 
more effective prevention system. As a 
contributor to the RAND working paper on child 
abuse and neglect prevention, Daro wrote, “the 
problem [of child abuse and neglect] and its 
solution are not simply a matter of parents doing a 
better job but rather creating a context in which 
‘doing better’ is easier” (Shaw & Kilburn, 2009, p. 

7). 

The Need to Focus on 
Infancy and Early 
Childhood 

The need for a Quality Improvement Center that 
would target early childhood and early 
intervention strategies, is essential. Daro, 
Barringer, and English (2009) pointed out, “A 
broad body of research has emerged which 
highlights the first three years of life as a 
particularly important intervention period for 
influencing a child’s trajectory and the nature of 
the parent-child relationship” (p. 1). Emphasis on 
attending to early childhood and early intervention 
includes focusing on the pregnant woman, the 
prenatal period, and the neonatal period (Daro, 
2009) to enable influences on parent behavior 
before neglectful and abusive patterns are 


established (Kotch, Browne, DuFort, Winsor, & 
Catellier, 1999). Practitioners also acknowledge 
the need to focus on infants and young children. 
For example, on-line survey respondents involved 
in the RAND Promising Practices Network 
project were asked about priorities in the child 
maltreatment field. With respect to the age group 
priority, 63% of the respondents indicated that 
children 0-2 should be the highest priority (Shaw 
& Kilburn, 2009). 

Rapid physical, cognitive, language, and social- 
emotional growth and development characterize 
the first few years in a child’s life. Focusing on 
the very youngest children recognizes the 
disproportionate rate at which the youngest 
children are maltreated. Wulzyn (2008) asserted, 
“More children start their child welfare careers 
during infancy than any other period within the 
span of childhood” (2008, p. 2). Although infants 
less than one year of age represent less than 6% of 
the child population they account for 10.6% of all 
substantiated abuse and neglect cases which 
represents a rate of 25.5 per 1,000 infants in the 
U.S. population (Centers for Disease Control, 
Spring 2009). The most recent data collected by 
Child Protective Services agencies through the 
National Child Abuse and Neglect Data System 
and reported in Child Maltreatment 2007, present 
the following national picture of the youngest 
victims of maltreatment (U.S. Department of 
Health and Human Services, Administration on 
Children, Youth and Families, 2009, p. 25, 27): 

■ Nearly 32% (31.9%) of all victims of 
maltreatment were younger than 4 years old. 

■ The youngest children had the highest rate of 
victimization. 
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■ Of the victims who were medically neglected, 
20.4% were younger than 1 year old. 

■ Victims who were younger than 1 year old 
comprised 12.0 % of all maltreatment victims. 

■ The rate of child victimization for boys in the 
age group of birth to 1 year was 22.2 per 
1,000 male children of the same age group. 

■ The child victimization rate for girls in the age 
group of birth to 1 year was 21.5 per 

■ 1,000 female children of the same age group. 

Infants and toddlers are not only the most 
vulnerable to the experience of maltreatment, they 
are also the most vulnerable to the effects of 
maltreatment. There is substantial research that 
shows a relationship between child maltreatment 
and a broad range of developmental problems that 
can have a life-long impact if not properly 
addressed (Lowenthal, 2000; Wiggins, Fenichel, & 
Mann, 2007). For example, research on the 
developing brain has provided extensive evidence 
that children’s earliest experiences create the brain 
architecture on which future development rests 
(Shonkoff & Phillips, 2000). Thus, everything 
regulated by the brain — including memory, 
emotions, and learning — is affected by the infant’s 
most immediate and influencing environments 
(Kotulak, 1996). 

We have mountains of research that tell 
us how children who’ve been exposed to 
abuse or neglect for long periods of time 
have less exploratory behavior. They 
don’t learn as well. They have more fears 
and they’re more likely to have physical 
and mental illnesses as adults. So abuse 
and neglect literally changes the brains of 
young children. It doesn’t just “affect 


their behavior.” It literally changes the 
wiring of their brains (Shonkoff, 2007). 

Maltreated infants and young children display a 
number of social-emotional disturbances, such as 
problems with forming secure attachments, 
specific mental health disorders including post- 
traumatic stress symptoms or disorder, adjustment 
disorders, regulatory disorders, and depression 
and withdrawal symptoms (Dubowitz, Papas, 
Black, & Starr, 2002; English et al., 2005). Long- 
term negative outcomes can include deficits in IQ 
scores and language ability; school failure; 
delinquency; high rates of unemployment and 
employment in low-level service jobs; and 
continuation of the cycle of violence (English, 
Widom, & Brandford, 2004; Thomas et al., 2003; 
Widon & Maxfield, 2001; Zolotor et al., 1999). 

The data that depict the national picture of the 
youngest victims of maltreatment, as well as the 
substantial research that shows a relationship 
between child maltreatment and a broad range of 
developmental problems, are extremely alarming. 
However, the good news is there is also strong 
evidence that “the course of development can be 
altered in early childhood by effective 
interventions that change the balance between risk 
and protection, thereby shifting the odds in favor 
of more adaptive outcomes” (Shonkoff & Phillips, 
2000, p. 32). 

The Need to Focus on 
High-Risk Populations 

The focus of the Q1C-EC is on infants and young 
children who are at high-risk for abuse, neglect, 
and abandonment. It should be noted that risk 
for child maltreatment crosses economic, racial, 


The Need for the Quality Improvement Center on Early Childhood: background Research and Evaluation Framework | 5 


ethnic, and geographic lines. Furthermore, a focus 
on vulnerable families does not obscure the 
recognition of and intent to focus on strengths 
within those same families. 

According to multiple sources, child neglect is the 
most prevalent form of child maltreatment in the 
United States (e.g., American Humane, 2009; 
Diaz, Peddle, Reid, & Wang, 2002; U.S. 
Department of Health and Human Services, 
Administration on Children, Youth, and Families, 
2008). For example, the most recent national data 
available on child maltreatment show that: 

■ In FY 2007, 59% of the children in the United 
States who were victims of abuse and neglect 
suffered from neglect alone; 1% were 
medically neglected. 

■ Of the victims who were medically neglected, 
20% were younger than 1 year. 

■ In FY 7 2007, 34% of child maltreatment 
fatalities in the United States occurred as a 
result of neglect only. 

Some researchers have suggested that the effects 
of neglect are more severe and enduring than 
those from abuse. For example, Perry, Pollard, 
Blakley, Baker, and Vigilante (1995) documented 
that the effects of early deprivation (i.e., neglect) 
on brain development were greater than those 
associated with trauma (i.e., abuse). Despite the 
widespread incidence of child neglect and its 
especially damaging effects, the overwhelming 
focus of child maltreatment theory, research, and 
practice is on child abuse. Researchers report that 
less is known about how to prevent neglect than 
other types of child maltreatment (DePanfilis & 
Dubowitz, 2005). One of the major challenges in 


studying child neglect is the nature of the risk 
factors. 

Many risk factors associated with families 
referred to CPS for neglect are also found 
in distressed, non-maltreating families. 
Families characterized by poverty are also 
characterized by stress and social 
isolation, but these families may not 
maltreat their children. The key to 
evaluating risk for neglect is to separate 
out those characteristics that may be 
common to many families and identify 
those who omit appropriate care for their 
children for reasons other than poverty 
(English, 1999, p. 202). 

Another major challenge relates to cultural 
differences in child rearing practices. 

Cross-cultural research has not yielded a 
universally ideal parenting strategy. What 
is considered optimal or deficient 
childrearing differs in various social and 
historical contexts. ... A challenge to a 
culturally informed understanding of child 
neglect is to distinguish clearly what 
portion of neglect is related to cultural 
differences and what to other multiple 
factors. This task is exceedingly complex 
and does not lend itself to easy solutions 
(Korbin & Spilsbury, 1999, p. 71, 82). 

In highlighting the need to focus on populations 
of highly vulnerable young children, the Children’s 
Bureau (2008, p. 5-6) pointed out: 

The Abandoned Infants Assistance Act, 
as reauthorized by the Keeping Children 
and Families Safe Act of 2003, highlights 
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the unique needs of a special population 
of vulnerable infants and young children. 
Studies have indicated that a number of 
factors contribute to the inability of some 
parents to provide adequate care for their 
infants and young children and that a lack 
of suitable homes have led to the 
abandonment of such children in 
hospitals for extended periods of time. 
Infants and children with life threatening 
conditions and other special needs, 
including those who are infected with 
HIV, those who have AIDS, and those 
who have been exposed to dangerous 
drugs, are at the greatest risk for 
abandonment and abuse or neglect and 
merit special attention. 


The Need for Effective 
Maltreatment 
Prevention 
Collaborations 

Researchers, practitioners, and policymakers agree 
that child maltreatment is much too complex for 
one organization, agency, or service system to 
successfully address on its own. Further, many 
children and families at high risk for maltreatment 
have a variety of physical, health, emotional, and 
educational needs. Thus, broad collaborations 
among key stakeholders are viewed as vital to the 
provision of needed services to children and 
families; to the success of child maltreatment 
prevention efforts; and to improved outcomes for 
children and families. Pollard (2005) described a 
fundamental issue regarding effective 
collaborations: distinguishing between 

coordination, cooperation, and collaboration. 
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Table 2: Pollard’s Distinction between “Coordination,” Cooperation,” and “Collaboration” 



Coordination 

Cooperation 

Collaboration 

Preconditions for 
Success 

("Must-Haves") 

■ Shared objectives 

■ Need for more than one 
person to be involved 

■ Understanding of who 
needs to do what by when 

■ Shared objectives 

■ Need for more than one 
person to be involved 

■ Mutual trust and respect 

■ Acknowledgment of 
mutual benefit of working 
together 

■ Shared objectives 

■ Sense of urgency and commitment 

■ Dynamic process 

■ Sense of belonging 

■ Open communication 

■ Mutual trust and respect 

■ Complementary, diverse skills and 
knowledge 

■ Intellectual agility 

Enablers (Additional 
"Nice to Haves") 

■ Appropriate tools (see 
below) 

■ Problem resolution 
mechanism 

■ Frequent consultation and 
knowledge- sharing 
between participants 

■ Clear role definitions 

■ Appropriate tools (see 
below) 

■ Right mix of people 

■ Collaboration skills and practice 
collaborating 

■ Good facilitator(s) 

■ Collaborative “Four Practices” mindset 
and other appropriate tools (see below) 

Purpose of U sing This 
Approach 

■ Avoid gaps and overlap in 
individuals' assigned work 

■ Obtain mutual benefit by 
sharing or partitioning work 

■ Achieve collective results that the 
participants would be incapable of 
accomplishing working alone 

Desired Outcome 

■ Efficiently-achieved 
results meeting objectives 

■ Same as for Coordination, 
plus savings in time and 
cost 

■ Same as for Cooperation, plus 

innovative, extraordinary, breakthrough 
results, and collective “we did that!” 
accomplishment 

Optimal Application 

■ Harmonizing tasks, roles 
and schedules in sifjiple 
environments and systems 

■ Solving problems in 
complicated environments 
and systems 

■ Enabling the emergence of 

understanding and realization of shared 
visions in complex environments and 
systems 

Examples 

■ Project to implement off- 
the-shelf IT application 

■ Traffic flow regulation 

■ Marriage 

■ Operating a local 
community-owned utility 

■ or grain elevator 

■ Coping with an epidemic or 
catastrophe 

■ Brainstorming to discover a dramatically 
better way to do something 

■ Jazz or theatrical improvisation 

■ Co-creation 

Appropriate Tools 

■ Project management tools 
with schedules, roles, 
critical path (CPM), PERT 
and GANTT charts 

■ "Who will do what by 
when" action lists 

■ Systems thinking 

■ Analytical tools (root cause 
analysis, etc.) 

■ Appreciative inquiry 

■ Open Space meeting protocols 

■ Four Practices 

■ Conversations 

■ Stories 

Degree of inter- 
dependence in 
designing the effort's 
work-products (and 
need for physical co- 
location of participants) 

■ Minimal 

■ Considerable 

■ Substantial 

Degree of individual 
latitude in carrying out 
the agreed-upon design 

■ Minimal 

■ Considerable 

■ Substantial 
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Pollard views “collaboration” as both structure 
and process. 

Collaboration entails finding the right 
group of people. . . ensuring they share 
commitment to the collaboration task at 
hand, and providing them with an 
environment, tools, knowledge, training, 
process, and facilitation to ensure they 
work together effectively. 
Cooperation and coordination are less 
elaborate and less ambitious collective 
undertakings (p. 1). 

Social service collaborations have a variety of 
names including integration of services, services 
integration, integrated services, array of services, 
systems of care, or services partnerships; Gardner 
and Young (2009) used the phrase “services 
partnerships.” They pointed out that while 

Overlapping risk factors establish a need 
for services partnerships, they also 
present major barriers to providing those 
services. These families are at times 
geographically distant from services, in 
denial about their need for services, and 
unable to overcome the chaos of their 
lives to keep appointments and participate 
actively in treatment. To the extent that 
“front end,” preventive services are 
voluntary, a significant portion of these 
families choose not to participate in 
voluntary services. . . . Harder-to-serve 
children and families by definition have 
greater needs, but are more likely to fail to 
benefit from services partnerships, or not 
even be able to enter such services (p. 2) 


Based on their research, Gardner and Young 
(2009) offered these recommendations for 
fostering effective maltreatment prevention 
collaborations for young children and their 
families: 

1 . Collectively clarify the nature of the 
collaboration, including the roles, 
responsibilities, and expectations of each 
partner. 

2. Collectively determine shared content and 
outcomes, in addition to shared processes, 
and how outcomes will be measured for their 
impact across agencies. 

3. Collaborations should occur at both the policy 
and practice levels. “Changes in practice 
without policy support will become isolated 
and partial; changes in policy. . . that do not 
reflect practice realities will remain paper 
decrees with little impact on children and 
families” (p. 4). 

4. Client engagement and retention must be 
major features of collaborations. 

5. Effective accountability mechanisms must be 
included in collaborations to assess which 
families are entering services and are being 
helped and which are not. 

6. The role of parents in several arenas related to 
service provision across the collaborating 
agencies is vitally important. 

7. “The content of services partnerships matters 
as much as the process. . . . (Services) must 
address (families’) co-occurring problems as a 
closely linked set, rather than as isolated 
categorical pieces” (p. 4). Screening tools 
should assess the co-occurrence of problems, 
rather than having families separately screened 
for different problems (e.g., substance abuse, 
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domestic violence) by different partnering 
agencies. 

8. “Integration in time matters more than 
integration of services at a single place, 
especially for younger children” (p. 5). That 
is, it is very important to share/ pass on the 
results of screening, identification, and 
intervention “from one service, stage, or 
developmental review to another” (p. 5). 

9. Providing staff training about how to 
effectively collaborate with others is an 
important intervention. “By itself, training 
does not change systems — it only equips 
workers with new skills. . . If the systems 
don’t change, the workers will find it difficult 
to use their new skills in their daily practice, 
and client outcomes are unlikely to improve” 

(P-5). 

10. The various costs of the services partnerships 
are vitally important and should be assessed, 
such as: what it will cost to sustain the 
partnership, how many children and families 
can be served within a given budget, what 
each added element will add to the total costs, 
and what savings are possible. 

1 1 . Collaborative service partnerships should 
consider both of the following approaches in 
order to be effective: “It takes data-driven 
accountability to bring partners to the table 
and prove to them that it is worthwhile to stay 
there and it takes strong relationships and 
trust built over time to keep the informal glue 
of the partnership working and to get past 
turf, barriers, and personal and institutional 
narrowness” (p. 9). 

Garder and Young (2009) also identified four 

challenges in determining the effectiveness of 

collaborations/ services partnerships: 


■ Ascertaining adequate baseline measures to 
determine whether the services partnerships 
made a difference. 

■ Measuring whether the short-term and 
intermediate outcomes are sustained over time 
and “whether dosage is adequate to make 
sustained impact an appropriate measure of 
the program’s effectiveness” (p. 4). 

■ Determining which of the services within the 
collaboration “creates value (and makes a 
difference) and which are just involving new 
players who may not be critical” (p. 4). 

■ Evaluating the “efforts to secure wider 
cooperation among a network of independent 

■ agencies” than simply evaluating a single 
agency’s “efforts to organize multiple services 
under a single roof’ (p. 4). 

Advancing a 
Promotion-Prevention 
Continuum Approach 

The growing body of research on developmental 
pathways and trajectories has contributed to 
reframing child maltreatment prevention efforts to 
include both prevention (i.e., reducing risk factors) 
and promotion (i.e., increasing protective factors) 
(e.g., DePanfilis & Dubowitz, 2005; Wulzyn, 
2008). 

Individual developmental pathways 
throughout the life cycle are influenced by 
interactions among risk factors that 
increase the probability of a poor 
outcome and protective factors that 
increase the probability of a positive 
outcome . . . Risk factors may be found 
within the individual (e.g., a 
temperamental difficulty, a chromosomal 
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abnormality) or the environment (e.g., 
poverty, family violence). Protective 
factors also may be constitutional (e.g., 
good health, physical attractiveness) or 
environmental (e.g., loving parents, a 
strong social network). The cumulative 
burden of multiple risk factors is 
associated with greater developmental 
vulnerability; the cumulative buffer of 
multiple protective factors is associated 
with greater developmental resilience 
(Shonkoff & Phillips, 2000, p. 30) 

Support for the promotion-prevention continuum 
approach was articulated by Surgeon General 
Richard Carmona in 2005: “I believe it is time for 
critical thinking to formulate a new national public 
health priority, preventing child maltreatment and 
promoting child well treatment” (p. 1). Over the 
past five years, the Strengthening Families 
Approach and Protective Factors framework have 
spurred a significant first step toward a national 
child maltreatment prevention approach based on 
promoting family strengths rather than focusing 
solely on reducing risk factors in individual 
families. A 2009 RAND Corporation study of the 
child abuse prevention field showed that the 
Strengthening Families Approach and Protective 
Factors Framework was the most recognized child 
maltreatment strategy even though it had only 
been developed in the past five years (Shaw & 
Kilburn, 2009). As this perspective has gained 
momentum, early childhood programs and other 


systems providing services for young children 
have come to see their roles as encompassing the 
prevention of child maltreatment as part of 
supporting optimal development. 

The QlC-EC’s promotion-prevention continuum 
approach complements the public health approach 
to violence prevention articulated in the Centers 
for Disease Control and Prevention child 
maltreatment prevention program coordinated by 
the Division of Violence Prevention (DVP). 
“DVP emphasizes efforts to prevent violence 
before it occurs. This requires not only reducing 
factors that put people at risk but also increasing 
the factors that protect people from becoming 
perpetrators of violence” (Centers for Disease 
Control and Prevention, n.d., p. 2). Many 
research studies have shown that experiences in a 
nurturing environment and consistent, positive 
relationships with family and caregivers create the 
best chance for optimal child development. As 
such, the DVP has adopted a strategy for child 
maltreatment prevention of promoting safe, 
stable, and nurturing relationships (SSNRs) 
between children and their caregivers (Centers for 
Disease Control and Prevention, n.d.). SSNRs are 
viewed as the positive end of this relationship 
continuum and, therefore, the antithesis of 
relationships and environments that foster the risk 
or occurrence of maltreatment, create childhood 
trauma, and compromise optimal child 
development, health, and well-being: 


Safe Relationships/Environments Neglectful and Violent Relationships/Environments 

Stable Relationships/Environments Unpredictable and Chaotic Relationships/Environments 

Nurturing Relationships /Environments .... Hostile/Cold or Rejecting Relationships/Environments 
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Designing the Delivery and Evaluation 
of an Intervention 


The Merits and 
Limitations of 
Randomized Control 
Trials 

There is widespread acceptance among 
maltreatment prevention stakeholders of the need 
for evidence-based practices (EBPs). The use of 
evidence-based practices is intended to promote 
the efficiency and effectiveness of practices by 
applying empirically supported principles (APA 
Presidential Task Force on Evidence-Based 
Practice, 2006). Nonetheless, the urgent need to 
identify effective child maltreatment prevention 
interventions and the push for evidence-based 
findings have generated much debate about the 
merits and limitations of the use of randomized 
controlled trials (RCTs) in child maltreatment 
prevention research. Some researchers support 
the notion of “a hierarchy of evidentiary rigor 
relating the design of a study to the confidence 
that could be placed in the findings, from the 
lowly, nearly valueless anecdote to the royalty of 
evidence, the RCT” (Berwick, 2008, p. 1182), 
while others take the position that RCTs are 
unworkable and irrelevant to real-world behavioral 
and psychosocial problems (Berwick). Daro (2009) 
asserted that this dichotomy creates an 
unnecessary false choice: 

A knowledge development model that 
places singular and preeminent emphasis 
on randomized clinical trials may be less 
appropriate today than in the past. . . A 
multifaceted lens is needed to provide 


policy makers and program planners the 
array of information necessary to identify 
and address the conceptual and adaptive 
changes facing efforts to expand and 
replicate community prevention efforts 
(p. 10). 

Research questions must guide the selection of 
scientific methods. Indeed, the appropriate match 
between research method and research question is 
a fundamental premise of scientific research. 

Critical research findings are not limited 
to those that result in statistically 
significant changes on various 
standardized measures of narrow 
constructs. Useful knowledge also is 
gleaned from the stories participants and 
providers tell in response to structured 
interviews.... To capture these diverse 
data, researchers need to implement 
diverse strategies (Daro, 2002, p. 8). 

Similarly, Rust and Cooper (2007) emphasized 
that researchers should, “Demand scientific rigor, 
but redefine scientific rigor to include new 
methods for measuring impact in complex, 
dynamic systems” (p. 110). Fundamentally, 

research projects must have designs that generate 
rigorous data about what works so that the 
findings may contribute to evidence-based or 
evidence-informed decision-making about 
maltreatment prevention practice and policy 
(Orszag, 2005). 


The Need for the Quality Improvement Center on Early Childhood: Background Research and Evaluation Framework | 12 


Defining Evidence- 
Based and Evidence- 
Informed Practice 

The definitions guiding the work of the Q1C-EC 
were developed by the Community Based Child 
Abuse Prevention Program (CBCAP) and 
Outcomes Workgroup, with support from the 
FRIENDS National Resource Center for CBCAP 
(2007). These definitions are consistent with 
those developed by the American Psychological 
Association and the Institute of Medicine (APA 
Presidential Task Force on Evidence-Based 
Practices, 2006; Sackett, Straus, Richardson, 
Rosenberg, & Haynes, 2000): 

Evidence-Based Practice: The 

integration of the best available research 
with child abuse prevention program 
expertise within the context of the child, 
family, and community characteristics, 
culture, and preferences. 

Evidence-Informed Practice: The use 

of the best available research and practice 


knowledge to guide program design and 
implementation within the context of the 
child, family, and community 
characteristics, culture, and preferences. 

Best available research refers to practices or programs 
that have been judged to be effective based on the 
quality, robustness, and validity of their rigorous 
scientific findings (Frieson & Cross, 2005). 
Prevention program expertise refers to the expertise of 
professionals or constituents “gained through 
observation, experience, reflection, and 
consensus” (Buysse & Wesley, 2006, p. 13). The 
context of the child, family, and community characteristics, 
culture, and preferences refers to the “unique 
preferences, concerns, and expectations” (Sackett, 
Straus, Richardson, & Haynes as cited in Buysse & 
Wesley, 2006, p. 13), “as well as local 
circumstances, values, and contexts that must be 
integrated into all practice decisions” (Buysse & 
Wesley, 2006, p. 13). 

The Children’s Bureau (CB) described the criteria 
within the continuum of evidence-informed to 
evidence-based programs and practices 
(Brodowski, 2008) as delineated in Table 3. 


Table 3: Continuum of Evidence-Informed to Evidence-Based Practices 


Emerging 

Promising 

Supported 

Well-Supported 

■ Ongoing collection of 
pre/ post data 

■All elements of “Emerging,” 
plus: 

■ All elements of 
“Promising,” plus: 

■ All elements of 
“Supported,” plus: 

■ Peer review 

■ 1 study, quasi-experimental 
design with control or 
comparison group 

■ 2 randomized clinical trials 
or 2 between group design 
studies (or comparable 
methodology) 

■ Multiple site replication 

■ Document all 

implementation activities 

■ Model fidelity 

■ One-year sustained effect 


Evidence-Informed 





^ .rwiaence-Dasea 
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Cultural 

Considerations and 

Evidence-Based 

Practice 

The influence of the culture of the participants in 
the design and delivery of maltreatment 
prevention strategies should not be ignored or 
minimized. It is arguable that the use of evidence- 
based practices could address the differential 
treatment and outcomes that diverse populations 
encounter within the child welfare system. 
“However, it is equally likely that EBPs could 
exacerbate and deepen existing inequities if they 
are implemented without sufficient attention to 
cultural competence and/or if policymakers fail to 
take into account the many practices within 
diverse communities that are respected and highly 
valued” (Isaacs, Huang, Hernandez, & Echo- 
Hawk, 2005, p. 4-5). Culture matters. 

Culture influences every aspect of human 
development.... Understanding this realm 
of influence is central to efforts to 
understand the nature of early experience, 
what shapes it, and how young children 
and the culture they share jointly 
influence each other over the course of 
development. . . . Given the magnitude of 
its influence on the daily experiences of 
children, the relative disregard for cultural 
influences in traditional child 
development research is striking 
(Shonkoff & Phillips, 2000, p.25). 

Integrating cultural considerations into program 
planning decisions must go beyond the typical 
“culturally sensitive” practices of: 


...Delivering services in a participant’s 
primary language, matching participants 
and providers on the basis of race and 
ethnicity, and incorporating traditional 
child rearing practices into a program’s 
curriculum. Far less emphasis has been 
placed on testing the differential effects of 
evidenced-based prevention programs on 
racial or cultural groups or the specific 
ways in which the concept of prevention 
is viewed by various groups and 
supported by their existing systems of 
informal support (Daro et al., 2009, p.l 1). 

Positive Self-Reflection 
as an Important 
Component of an 
Intervention 

Positive self-reflection is the process of examining 
the content and impact of one’s personal values, 
beliefs, styles of communication, and experiences 
by posing positive questions that focus on success. 
Daro et al. (2009) suggest child maltreatment 
prevention programs should examine “the specific 
way in which their innovation or strategy will 
strengthen a parent’s ability for self-reflection in 
discerning appropriate options for themselves and 
their children” (p. 14). Positive self-reflection is 
considered to be a critical component of a 
maltreatment prevention intervention in that 
sociological research has shown that when people 
“focus on human ideals and achievements, peak 
experiences, and best practices, these things — not 
the conflicts — tend to flourish” (Mohr & Watkins, 

2002, p. 2). 
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Complex Adaptive 
Systems 

The focus on research in the child abuse and 
neglect prevention field is expanding from looking 
at individual programs and services to include a 
broader, comprehensive focus on systems and the 
complexity within which programs function. This 
does not mean that programs and services are 
unimportant, but that attention to the parts 
(programs) alone is insufficient. Attention is 
needed to the parts, the whole, and the greater 
whole. A systems orientation is valuable to 
support this expanded thinking. 

’’Systems” are defined as the parts and 
interconnections that form a coherent whole. 
Programs and services are considered within their 
context, paying attention to how the 

interconnections, relationships, and differences 
that exist among and between parts form a 
coherent whole. Although there are many issues 
related to a systems orientation that are important, 
two primary ones are the distinction between 
nested and networked social systems and a 
recognition of different dynamics within systems. 

Social systems tend to be conceptualized in two 
ways — as either nested or networked. Most 
bureaucratic, hierarchical organizations build on a 
notion of nested systems where one level is nested 
within another. For example, a county agency may 
be nested within a state system and subject to the 
policies and direction of a state agency. In today’s 
complex world, networked systems are 
increasingly and especially important. Partnerships 
and interconnections among service providers 
across and among organizations are examples of 
networked systems. Thinking in terms of a 


networked model of systems directs attention to 
the complex relationships that can exist across and 
among levels of the ecological model. 

From Knowledge 
Development to 
Dissemination and 
Integration 

Merely generating rigorous data is not sufficient 
for improving practice or shaping policy. Equally 
important as seeking clarity about the nature of 
research evidence and appropriate research 
methods are concerns about (a) disseminating 
research findings, (b) contextualizing and 
translating research findings into practice and 
policy, (c) utilizing research findings, and (d) 
making policy recommendations and influencing 
proactive change in the general public’s attitudes 
about child abuse and neglect. 

Understanding how best to disseminate and 
integrate new knowledge about effective 
prevention strategies is keenly important, but 
research in this area is substantially limited. 
“Identifying and implementing efficient and 
effective methods for dissemination and 
implementation is critical in optimizing the access 
and reach of evidence-based programs to prevent 
child maltreatment and promote safe, stable, and 
nurturing relationships” (Centers for Disease 
Control and Prevention, 2009, p. 78). Daro et al. 
(2009) emphasized: 

It is not enough for scholars and program 
evaluators, on the one hand, to learn how 
maltreatment develops and what 
interventions are effective and for 
practitioners, on the other, to implement 
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innovative interventions in their work 
with families. Instead, initiatives must be 
implemented and assessed in a manner 
that maximizes both the ability of the 
researchers to determine the effort’s 
efficacy and the ability of program 
managers and policy makers to draw on 
these data to shape their practice and 
policy decisions (p. 11). 

A burgeoning field of research investigates how 
new knowledge is integrated into practice in 
complex settings. The literature includes research 
and theory about the diffusion of innovation, 
organizational learning, knowledge management, 
learning theory, and ways of structuring learning 
in complex systems. Models of how to move 
from knowledge development and dissemination 
to knowledge integration in complex systems are 
in the early stages of development and are 
emerging in many fields such as business, 
sociology, development, education, and public 
health (see Lubeck & Post, 2000; O’Dell & 
Grayson, 1998; Ordonez & Serrat, 2009; Parsons, 
2009; Rogers, 2003; Senge, 1990; Senge, Kleiner, 
Roberts, Ross, & Smith, 1994; Wenger, 1999; 
Wenger, McDermott, & Snyder, 2002; Wenger & 
Snyder, 2000). 

Tanzer (2009) addressed how to more effectively 
frame the child abuse prevention message and 
engage policymakers, prevention partners, and the 
general public in prevention work. He 
extrapolated the following “deficiencies in public 


understanding” (p. 8) about child abuse and 
neglect prevention from a report commissioned 
by Prevent Child Abuse America; these 
deficiencies also support the need for reframing 
the prevention message. 

1 . The cause of child abuse and neglect is due to 
bad parenting, bad people (e.g., “stranger 
danger”), or the failure of Child Protective 
Services. 

2. The family is an autonomous unit. Thus, 
child rearing takes place in a private space so 
outside intervention is equivalent to 
interfering with family affairs. 

3. Lack of understanding about how children 
develop and how the effects of abuse and 
neglect can be overcome through intentional 
effort. 

4. Lack of understanding about the role of the 
broader society in prevention efforts. 

5. Feelings of inadequacy to competently 
address the problem of child abuse and 
neglect which are reinforced when 
government/ outside forces begin to address 
the problem. 

6. The message is overshadowed by 
sensationalized extreme cases of child abuse 
and neglect. 

Table 4 provides a summary of Tanzer’s (2009) 
recommendations for engaging policymakers, 
prevention partners, and the general public in the 
work of child abuse and neglect prevention. 
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Table 4: Recommendations for Engaging Stakeholders in Child Maltreatment Prevention Work 


Stakeholders 

Recommendations 

State and Federal 
Policymakers 

■ Promote a prevention policy agenda that focuses on enhancing child development, engaging 
communities, supporting families, and preventing child abuse and neglect before it occurs 

■ Articulate the impact of not preventing child abuse and neglect and explain the return on investment 
in preventing child abuse and neglect 

■ Nurture a diverse pipeline of champions and provide them with solid evidence 

Prevention Partners 

■ Provide partners with evidence about what works and the tools for implementation 

■ Develop prevention leaders who can advance the prevention message 

■ Help partners to enhance their advocacy, organizational, and communication skills 

■ Determine the central prevention theme (s)/ value (s) that will be promoted 

■ Expand the pool of partners to include diverse fields/ disciplines 

General Public 

■ Focus prevention efforts on enhancing child development, engaging communities, supporting 
families, and preventing child abuse and neglect before it occurs 

■ Articulate the impact of not preventing child abuse and neglect and explain the return on investment 
in preventing child abuse and neglect 

■ Build on existing networks 

■ Clearly articulate the roles the general public can play 


Characteristics of 
Effective Maltreatment 
Interventions 

Daro et al. (2009, p. 2) reviewed child 
maltreatment intervention programs for children 
birth-5 that met three important criteria: 

• Programs had to reflect relevant theory that 
draws on a descriptive etiologic framework. 

• Programs had to be evidence-based, 
demonstrating significant results in the core 
domains of interest (e.g., promoting optimal 
child development, increasing protective 


factors, reducing risk and preventing child 
maltreatment). 

• Where applicable, programs had to be rated as 
“promising” or “proven” by at least one 
independent review system. 

Overall, the researchers identified several common 
characteristics of the effective intervention under 
review: 

1 . Theoretical integrity: “All follow a clear logic 
model: definition of the problem, examination 
of etiology and context, identification of 
measureable goals, and construction of an 
intervention with a cohesive structure. . . The 
pivotal element for success was not the 
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effective execution of individual program 
components but rather the conceptual 
framework on which the program rests” (p. 

4). 

2. Intervention targets the earliest stages: Since 
learning begins at birth, effective interventions 
focus on maximizing a child’s developmental 
potential by offering assessment and support 
starting during pregnancy, immediately after 
birth, or during pre-school. 

3. Impact the bi-directional interaction between 
individuals and their families: “Successful 
programs approach prevention with the view 
that both children and parents (as individual 
actors) and the family (as a cohesive unit) 
should be served by interventions” (p. 5) 

4. A multi-tiered program structure: “Many 
proven/promising programs stagger services 
so that those most in need receive an 
intensive level of service, while those with less 
need receive a decelerated level of service.... 
Ultimately, a staggered program design can 
contribute to greater program efficacy, 
efficiency, and cost-effectiveness, and is 
consistent with the public health model of 
‘minimal sufficiency” (p. 5). 

5. Supplement and link prevention programs to 
the existing local network of social support 
services: “By conceptualizing programs as 
new components within a preexisting system, 
programs developers can enhance both the 
potential impacts of their own efforts as well 
as increase the probability that these impacts 
will be sustained over time” (p. 6). 

6. Systematically examines the needs of the 
target population across a number of 
domains: The effects of many successful 
interventions “often fade over time in part 
because local communities and public 


institutions fail to reinforce the parenting 
practices and choices these programs 
promote. They also fade because too much 
emphasis has been placed on the structure 
and content of the intervention and too little 
emphasis has been placed creating a 
mechanism within families as well as 
organizations to effectively discern their needs 
and efficiently utilize those resources that are 
available to them” (p.13). 

7. Builds relationships: “The strength and 
quality of the participant-provider relationship 
is often viewed as one of the most, if not the 
most, important determinants of proximate 
and distal outcomes. Personal contact is 
certainly a key feature of successful programs, 
particularly with families who are extremely 
isolated and disconnected from formal and 
informal supports” (p. 11-12). 

8. Builds a context that offers ongoing support 
and access to other interventions: “Many of 
the most successful programs offered a variety 
of service components, including child 
development (e.g., home visits). . . family 
development (e.g., comprehensive health and 
mental health services) . . . and community 
building” (p. 6). 

Challenges Facing the 
Maltreatment 
Prevention System 

There are numerous challenges facing the child 
maltreatment prevention system (Goldman, Salus, 
Wolcott, & Kennedy, 2003). Right now, the gulf 
between what is known about effective prevention 
programs and implementation of prevention 
programs remains wide. Research on effective 
primary and secondary prevention programs for 
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young children at risk of child welfare 
involvement and their families points to critical 
risk factors such as poverty and caregiver 
interpersonal violence. Largely absent from this 
research are effective strategies to address neglect 
(Klevens & Whitaker, 2007). Among secondary 
prevention strategies, researchers identify the 
importance of parent-child interaction and 
environmental factors (Geeraert, den Noortgate, 
Grietens, & Onghena, 2004). These factors are 
further compromised by parental substance use 
(Eiden, Leonard, Hoyle, & Chavez, 2004). 
Although many prevention programs address 
these factors, their implementation with 
consistency and fidelity is limited, they are not 
widely available, and there is too little attention to 
outcomes (Geeraert et ah). 

Research suggests that while some policies are 
supportive of prevention efforts, there are also 
many policy-linked limitations that restrain more 
comprehensive approaches. For example, despite 
research that demonstrates the efficacy of mental 
health interventions for young children, some 
policies undermine attempts to make these 
interventions widely available. Certain state and 
federal fiscal policies require a mental health 
diagnosis in order for young children to access 
needed mental health services, while others do not 
reimburse for services provided in non-office 
based settings, such as child care centers (Cooper, 
2008). 

An often overlooked challenge to prevention 
programs relates to the individuals implementing 
the program; that is, identifying professionals who 
(a) have the ability to demonstrate unconditional 
positive regard and respect for the individuals and 
families they serve; (b) believe that individuals 
have inner strengths that can facilitate change; and 


(c) are willing to customize services and service 
delivery approaches to reflect the characteristics, 
culture, context, strengths, and needs of each 
family (Goldman et al., 2003). Few studies have 
examined the impact of these and other factors 
such as program quality assurance and participant 
satisfaction on the effectiveness of prevention 
approaches. 

Other overlooked factors that contribute to the 
effectiveness of the delivery of a maltreatment 
prevention intervention are staff preparation and 
organizational capacity. Elliott and Mihalic (2004) 
pointed out that even when programs have robust 
designs and rigorous evaluation plans, they may 
not have the organizational capacity to effectively 
deliver the program, such as marketing capability, 
technical assistance, and a data management 
system. Similarly, Daro et al. (2009) addressed the 
importance of professional development vis-a-vis 
the successful implementation of a prevention 
program: 

When new practice reforms are 
introduced at an agency, staff need to be 
given sufficient time to work with the 
model and build confidence in their 
ability to deliver the intervention with 
fidelity. Similarly, management. . . needs 
to consider how best to orient staff to the 
new component and its relationship to 
other programs operate by the agency. It 
is important. . . to place a higher level of 
priority on developing the workforce and 
creating strategic plans for training and 
development. This will improve the 
ability of the organization to sustain 
robust services” (Daro et al., 2009, p. 8). 
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The Strengthening 
Families Approach & 
the Protective Factors 
Framework 

Typically, families are targeted for child abuse and 
neglect prevention programs on the basis of 
various risk factors known to be correlated with 
child maltreatment such as low maternal age, 
presence of substance abuse in the household, and 
low income (Thomas et al., 2003). Singularly 
focusing on risk factors to identify families seems 
appropriate if the goal is to provide services to 
families most in need, but this strategy has several 
key drawbacks. 

First, the prediction of which families may 
maltreat their children on the basis of identified 
risk factors is relatively unreliable. The notion of 
“risk” itself implies both an increased likelihood 
that maltreatment may occur due to various 
factors and the possibility of variability in reaction 
to the same factors (Fraser, Kirby, & Smokowski, 
2004). This suggests that many families with 
child, parent, family, or community risk factors do 
not actually maltreat their children; other factors 
operate to mitigate their risks (Fraser, 2004). 

Second, some of the risk factors such as maternal 
age or a premature birth, are static, so they are not 
amenable to an intervention’s influence (Ross & 
Vandivere, 2009). Thus, the program’s strategies 
can only have limited impact on reducing the 
overall risk for a given family. 

Third, targeting families according to risk factors 
may have the unintended effect of discouraging 
them from participating; families do not want to 
be labeled as “high-risk” or potential child 


abusers. This stigmatization no doubt contributes 
to the difficulty that many prevention programs 
experience in recruiting families and keeping them 
engaged once they are enrolled in the program 
(Daro et al., 2009; Daro & Donnelly, 2002; Olds 
& Flenderson, 1991). The challenge is to 
normalize prevention strategies so that needs are 
assessed and relevant supports are provided to all 
families served (Daro & Donnelly). 

Fourth, Kirk, Firman, & Baker (2004) pointed out: 

Proving a negative, in this case proving 
that child maltreatment did not occur 
because of the specific program or service 
is, if not an impossible task, an extremely 
difficult one. . . 

However, a program that facilitates 
positive change in risk factors increases 
the likelihood of greater safety for 
children (p. 10). 

Finally, an exclusive or primary focus on risk 
factors may interfere with engaging a broad array 
of partners in child abuse prevention. The 
orientation of many child and family serving 
programs is to promote healthy physical, social, 
emotional, and cognitive development; to enhance 
their children’s early experiences; and to approach 
families from a proactive perspective, rather than 
a deficit/ risk-based perspective. This inherent 
strengths-based orientation is conducive to 
engaging programs around a resiliency framework 
and helping them to see how their work can be 
effective in preventing child maltreatment. 

Both researchers and practitioners are aware that 
reducing risks is not enough to ensure that infants 
and young children in vulnerable families are on a 
trajectory to optimal development. Identifying 
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and understanding protective factors are equally as 
important as researching risk factors; but 
protective factors have not been studied as 
extensively or rigorously as risk factors (Centers 
for Disease Control and Prevention, 
http:/ / www.cdc.gov/ print. 

do?url=http%3A/ / www. cdc.gov/ ncipc / dvp/ 
CMP/ CMP-risk-p-factors.htm). 

■ Risk factors are defined here as conditions or 
attributes of individuals, families, 
communities, or the larger society that 
increase the probability of maltreatment and 
poor outcomes. 

■ Protective factors are defined here as 
conditions or attributes of individuals, 
families, communities, or the larger society 
that decrease the probability of maltreatment 
and increase the probability of positive, 
adaptive, and resilient outcomes even in the 
face of risk factors. (Fraser et al., 2004; 


Shonkoff & Phillips, 2000; Thomas et al., 
2007). 

The focus of the QIC-EC is on the five protective 
factors articulated in the Strengthening Families 
Approach: parental resilience, social connections, 
knowledge of parenting and child development, 
concrete support in times of need, and social and 
emotional competence in children. Although the 
framework assumes that the ability to form a 
warm, secure bond with a young child may be 
regarded as a component of “parental resilience” 
and of “building social emotional competence in 
children,” for the purposes of the QIC-EC 
research, “nurturing and attachment” will be 
treated as a sixth interrelated protective factor. 
More empirical evidence is needed about the 
processes and outcomes of systematically building 
these Protective Factors in families at high risk for 
child maltreatment. 


Table 5. The Strengthening Families Protective Factors, Plus One 


Protective Factors 

Definitions 

Parental Resilience 

The ability to establish positive relationships, including attachment to a child; capacity 
to cope with stresses of daily life and recover from challenges. 

Social Connections 

Having friends, family members, neighbors, and others who provide emotional support 
and concrete assistance to parents. 

Knowledge of Parenting and 
Child Development 

Having accurate information about child development, appropriate developmental 
expectations, and knowledge of alternative discipline techniques. 

Concrete Support in Times of Need 

Having financial security to cover basic needs and unexpected costs; formal supports 
like TANF, Medicaid and job training; crisis services including mental health, domestic 
violence and substance abuse. 

Children’s Social and 
Emotional Competence 

A child’s ability to interact positively with others and communicate his or her emotions 
effectively. 

Nurturing and Attachment 

The ability to respond appropriately, warmly, and consistently to the basic needs of 
infants and young children and to foster a strong and secure parent-child attachment. 
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The Overarching Research Question for 
the R&D Projects 


The overarching research question for the R&D 
projects funded by the Q1C-EC is: 

How and to what extent do collaborations 
that increase protective factors and 
decrease risk factors in core areas of the 
social ecology result in optimal child 
development, increased family strengths, 
and decreased likelihood of child 
maltreatment within families of young 
children at high risk for child 
maltreatment? 

The Four Core Areas 
of the Social Ecology 
for the R&D Projects 

The QIC-EC Team has identified a core area at 
each level of the social ecology as leverage points 
or areas of change within which R&D projects 
could focus their interventions: primary caregiver and 
target child (individual level), social support 
(relationship level), community connections 
(community level), and public polity and social norms 
(systems level). 

Primary Caregiver and Target 
Child (Individual Level) 

Given the enormous amount of research that 
demonstrates that “early experiences matter” in all 
domains of development, the nature of a 
caregiver’s knowledge, attitudes, behaviors, skills, 
capacities, and psychological functioning, takes on 


great importance. Similarly, characteristics, 
attributes, and capacities of infants and young 
children influence how adults respond to them, 
and consequently how they grow, develop, and 
learn. The outcomes of focus for the R&D 
projects are directly tied to the primary caregiver 
and target child: optimal child development, 
increased family strengths, and decreased 
likelihood of child maltreatment. 

Social Support (Relationship 
Level) 

In this context, social support refers to those 
people who most closely surround and are 
involved with families and young children; who 
may have direct, regular contact with the child, 
often serving as caregivers while parents work, go 
to school, or engage in other activities; and who 
are sometimes living in the same household as the 
parent(s) and child. Individuals who serve as 
social supports — mothers, fathers, grandparents, 
other relatives, friends, and even co-workers — 
provide advice and resources about parenting and 
child rearing, transmit cultural values and 
practices, and engender feelings of connectedness 
and security. Several researchers have found that 
social support may serve as a buffer against life 
stressors for both the parents and children 
(Morisset, 1993; Jarrett, 1995). 
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Community Connections 
(Community Level) 

The notion of community connections grows out 
of the idea that “the family is nested in a 
neighborhood system that provides support, or 
fails to provide support, for child rearing” (Fraser, 
Kirby, & Smokowski, 2004, p. 44). The work of 
Sampson, Raudenbush, and Earls (1997) suggests 
that the presence and involvement of supportive 
others outside of one’s family and close friends 
may help to promote optimal child development 
even in the face of poverty and other community- 
level risk factors. In the context of the R&D 
projects, community connections include several 
key components within a community that may be 
engaged to help build protective factors and to 
help identify challenges that may create risk 
factors or interfere with the reduction of risk 
factors: 

■ Community leaders 

■ Organizations (e.g., faith-based organizations, 
parent organizations) 

■ Neighbor alliances (e.g., neighborhood 
associations, neighborhood watch groups) 

■ Formal support programs and service 
providers (e.g., early care and education 
centers; recreational facilities; local health, 
mental health, and social services) 

The community connections core area also 
includes “social cohesion,” defined as: “the degree 
to which members of a neighborhood share 
values, beliefs, and expectations and the degree to 
which neighbors are willing to take action on 
behalf of others” (Fraser, Kirby, & Smokowski, 


2004, p. 44); “the degree to which (members of a 
neighborhood) feel their neighbors could be 
counted on to help each other or could be 
trusted” (Daro & Donnelly, 2002, p. 442). 

Public Policy and Social Norms 
(Systems Level) 

For the purpose of the R&D projects, public 
policy refers to: 

A course of action or inaction chosen by 
public. ..authorities to address a problem. 
Public policy is expressed in the body of 
laws, regulations, decisions (including 
funding decisions), and governmental 
action. . . . These policies can be critical in 
shaping the environment in which child 
maltreatment occurs (Centers for Disease 
Control and Prevention, 2009, p. 78). 

Public policies should be examined to determine if 
they are effective in strengthening families and 
preventing child maltreatment. Similarly, the 
larger culture, as expressed in social norms, plays a 
significant role in how families care for their 
children. Given national statistics on such factors 
as the number of children who live in poverty, the 
incidence of intimate partner violence, the amount 
of media violence, the reliance on corporal 
punishment, and the maltreatment rates of infants 
and young children, it is important to examine and 
identify strategies to change social norms that 
reflect a societal acceptance of violence and place 
a low value on children. The suggestions put 
forth by Tanzer (2009) offer some suggestions for 
beginning this process. 
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Outcomes, Indicators, Measurement, 
and Instrumentation 


Outcomes and 
Indicators 

The three outcomes of focus for all Q1C-EC 
R&D projects are: optimal child development, 
increased family strengths, and decreased 
likelihood of child maltreatment. 

■ The outcome “optimal child development” 
will be measured by pre- and post- 
intervention assessments of the “child well- 
being” domain, to include health, 
education/ cognitive well-being, and social- 
emotional well-being indicators. 

■ The outcome “increased family strengths” will 
be measured by pre- and post-intervention 
assessments of the following domains and 
indicators: “home and community” (home 
safety and social connectedness); “parent 
capacity” (parenting skills, parenting 
knowledge of child development, and parent 
mental health); “substance abuse” (type, 
frequency, and problem behaviors associated 
with risky substance abuse; participation in 
substance abuse treatment programs); 
“financial solvency” (income; housing 
stability; food security) and “family conflict” 
(types and levels of family conflict). 

■ The outcome “decreased likelihood of child 
maltreatment” will be measured by pre- and 
post-intervention assessments of the balance 
between protective factors and risk factors. 
Assessment of risk and protective factors will 
be conducted. 


Measuring the 
Outcomes 

A multifactorial approach to assessment will be 
used by the R&D projects in accordance with 
Meisels’ assertion that “the more sources of data 
that are tapped, the more adequate and useful will 
be the conclusions drawn from the assessment” 
(1992, p. 4). Even though the R&D projects have 
time limitations and all of the concerns about the 
primary caregiver, child, social support, and 
community connections that might arise from the 
multiple assessments cannot be addressed, 
multifactorial assessments enable data-driven and 
prioritized intervention plans that produce richer 
evaluation data. 

Further, in keeping with the QIC-EC’s perspective 
on evidence-based practice, professional judgment 
that is informed by knowledge of the context of 
the child, family, community characteristics, 
culture, and preferences should be factored into 
the interpretation of assessment results and 
intervention plans and decisions. This is 
particularly important when using standardized 
instruments that did not include culturally diverse 
populations in the norming process and when 
cultural or community characteristics should be 
factored in risk assessments. 

Selecting Common 
Instruments 

R&D projects will be required to use several 
common instruments to support the cross-site 
evaluation. Some will be standardized assessment 
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tools and others will be developed by the QIC- 
EC. Several criteria were used in recommending 
common standardized instruments. They should: 
(a) include measures for assessing infants birth-24 
months; (b) not require special credentials for 
administration, scoring, and interpretation; (c) be 
easily obtainable; (d) be designed to measure 
multiple indicators related to the R&D project 
outcomes; (e) have a theoretical orientation that 
was empirically-based; and (f) have acceptable 
validity and reliability data. Preference was also 
given to assessment tools that have Spanish 
language versions and that actively engage parents 
in order to encourage self-reflection. 

Three of the recommended common instruments 
are not standardized as they are currently under 
development by the Q1C-EC Team: a 

demographic data tool template, the protective 
factors assessment tool, and the Social Support 
Map. While numerous instruments are described 
here as prospective tools for assessing indicators 
of the three outcomes, this does not mean that the 
QIC-EC Team is recommending use of all the 
instruments for each outcome; all listed simply are 
regarded as viable options. The information that 
follows provides an overview of recommended 
common instruments. 

A Demographic Data 
Tool Template 

A template for recording common demographic 
and other data on all participating primary 
caregivers, gathered via an in-person interview, is 
currently under development by the QIC-EC 
Team. This instrument will include items that 
assess family composition, neighborhood context, 
and other indicators suggested by Child Trends as 


most easily assessed in this format (e.g., frequency 
and types of “junk” food eaten, frequency of 
doctor visits, number of emergency room visits 
for injuries, service access, and reasons for 
accessing or not accessing services). The purpose 
of the template is to ensure that certain variables 
are recorded on all participating families. Funded 
R&D projects may choose to add demographic 
information to the template. 

Measuring the 
Protective Factors 

An extensive protective factors assessment tool is 
currently under development by the QIC-EC. 
This tool will be used by funded R&D projects as 
a pre-, post-, and repeated measure instrument to 
gather baseline, intermediate, and final data on 
participating families’ knowledge, attitudes, and 
behaviors that support and reflect the identified 
protective factors. It might not be necessary to 
focus on each protective factor with each 
participating family. Daro et al. (2009) pointed 
out: 

The most salient protective factors or risk 
factors to target to avoid negative 
outcomes will vary across populations as 
well as communities. Finding the correct 
leverage point or pathway for change for 
a specific family requires careful 
assessment followed by an offer of 
assistance commensurate with a family’s 
level of need. (p. 10). 

Measuring Risk Factors 

Numerous risk assessment instruments have been 
developed for child protection work to gauge the 
potential occurrence or reoccurrence of child 
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maltreatment. The risk assessment instruments 
recommended for common use by all funded 
R&D projects are the Child at Risk Field, the 
Childhood Level of Living Scale, and the Child 
Abuse Potential Inventory. 

The Child at Risk Field 

The Child at Risk Field (CARF) is designed to 
measure risk factors for all types of maltreatment. 
The CARF consists of 14 open-ended questions 
to rate risk influences for the child, parent, family, 
maltreatment, and the level of intervention needed 
to address the risk factors. Four additional 
qualifiers are included that measure the duration 
and pervasiveness of negative influences that may 
contribute to risk of maltreatment and the 
caregiver’s acknowledgement and control of these 
negative influences. Practitioners rate information 
on a risk scale ranging from 0 — 4. A final risk 
score is determined by combining the average 
ratings on the 14 risk factors and 4 qualifiers and 
dividing by 2. Interpretations of risk scores range 
from “no likelihood” to a “high likelihood of 
maltreatment.” (Source: Milner, Murphy, Valle, & 
Tolliver, 1998) 

The Childhood Level of Living Scale 

The assessment of risk for neglect is essential 
given the data on the vulnerability of infants and 
young children to the experience of neglect. 
However, the assessment of risk for or actual 
neglect is difficult because the practitioner has to 
be sensitive to questions that impose middle-class 
values on a client or require resources beyond that 
of the client. The Childhood Level of Living Scale 
(CLLS) is one of the few rating scales designed to 
assess positive and negative elements of child care 
and home life particularly with regard to neglect of 


young children under age 7. In this regard, the 
scale focuses on major areas of concern relative to 
the child’s minimal health, safety, and stimulation 
requirements. The CLLS contains 99 items 
organized into two scales: Physical Care and 
Emotional/Cognitive Care. The Physical Care 
scale includes items that comprise five subscales: 
General Positive Care, State of Repair of House, 
Negligence, Quality of Household Maintenance, 
and Quality of Health Care and Grooming. The 
Emotional/ Cognitive scale includes items that 
comprise four subscales: Encouraging 

Competency, Inconsistency of Discipline and 
Coldness, Encouraging Superego Development, 
and Material Giving. (Source: Pecora, Fraser, 
Nelson, McCroskey, &Meezan, 1995) 

The Child Abuse Potential Inventory 

The Child Abuse Potential Inventory (CAP or 
CAPI) is a widely used self-report tool that was 
designed to measure both personal and 
interactional dimensions of parenting in order to 
assess the risk of perpetrating physical abuse. The 
CAP inventory is a 160-item questionnaire in an 
agree/ disagree response format. The primary 
scale is a physical abuse scale and seven factor 
scales grouped as personal (Distress, Rigidity, 
Unhappiness, and Loneliness) and interpersonal 
(Problems with Child and Self, Problems with 
Family, and Problems from Others). The CAP 
takes 15-20 minutes to administer, requires a 
third-grade reading level, and is available in a 
Spanish language version. Extensive research with 
the CAP has revealed scores that correlate with an 
adult’s childhood history of abuse, life stress, and 
psychological traits such as external locus of 
control, low self-esteem, high reactivity, 
ineffective coping, and depression. (Source: 
Pecora et al., 1995) 
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Measuring Optimal 
Child Development 

The outcome “optimal child development” will be 
measured by pre- and post-intervention 
assessments of the domain “child well-being” 
identified by Child Trends to include the 
indicators health, education/ cognitive well-being, 
and social-emotional well-being. Some of the 
instruments for making direct assessments of child 
well-being identified by Child Trends are included 
here. Health indicators will be assessed in an in- 
person interview using the demographic data tool, 
per the recommendation of Child Trends. 

As the initial target population is children ages 
birth— 24 months, two common instruments are 
recommended for this age group: Ages and Stages 
Questionnaire, Third Edition and the Denver 11. 
Four other instruments are recommended for use 
with infants and young children 12 months and 
older: the Infant Toddler Social-Emotional 

Assessment, the Child Behavior Checklist for 
Ages 11/2—5 and the Child Development 
Inventory, and the Childhood Level of Living 
Scale. 

Ages and Stages Questionnaires, 
Third Edition (ASQ-3) 

The Ages and Stages Questionnaire, Third Edition 
(ASQ-3) is a parent-completed developmental 
screening for infants and young children ages 1-66 
months. The areas screened include 
communication, gross motor, fine motor, problem 
solving, and personal-social. The ASQ-3 includes 
a series of 21 questionnaires based on age intervals 
by months: 2, 4, 6, 8, 9, 10, 12, 14, 16, 18, 20, 22, 
24, 27, 30, 33, 36, 42, 48, 54, and 60. There are 30 
items per questionnaire with six items for each of 


the five areas screened. The ASQ-3 requires a 4 th 
to 6 th grade reading level and can be completed by 
most parents in 10-15 minutes; scoring takes 2-3 
minutes. Raw scores for each area are calculated 
and compared to cutoff points for each area, 
which show whether the child’s development is 
delayed (i.e., refer for assessment), suspect (i.e., 
monitor and rescreen in 2—6 months), or typical. 
The ASQ-3 has both English and Spanish 
language versions. 

(Source: 

http://www.myexpospace.com/DLQA/PAS%20 
2009/ file/ASQ-3_at_a_glance.pdf) 

Denver II 

The Denver II, a revision of the Denver 
Developmental Screening Test, is designed for use 
by professionals or trained paraprofessionals to 
monitor the development of infants and young 
children, 2 weeks to 6 years old. The Denver II 
has 125 items including a set of questions for 
parents and simple tasks for children on that 
cover four general areas: 25 personal-social items 
(such as smiling), 29 fine motor adaptive items 
(such as grasping and drawing), 39 language items 
(such as combining words), and 32 gross motor 
items (such as walking). The number of items 
administered during an assessment will vary with 
the child’s age and ability. 

The Prescreening Developmental Questionnaire 
(PDQ-1I) was developed to help parents quickly 
identify whether their children need further 
assessment. The PDQ-II is a pre-screening 
consisting of 91 parent questions extrapolated 
from the DDST-II. It was created for parents to 
complete easily and quickly to assess whether their 
children have non-normal scores and need to 
complete the full DDST-II. The Denver II and 


The Need for the Quality Improvement Center on Early Childhood: Background Research and Evaluation Framework | 27 


PDQ II have both English and Spanish language 
versions. 

(Source: 

http://www.denverii.com/ denveriiarticle.html) 

The Infant Toddler Social 
Emotional Assessment (ITSEA) 

The Infant Toddler Social Emotional Assessment 
(ITSEA) is a tool designed to measure strengths 
and weaknesses related to social-emotional 
development and competencies in children ages 
12 to 48 months old. It may also be used to 
identify caregivers who could benefit from 
additional dialogue about children’s social- 
emotional developmental expectations. The 
ITSEA relies on parents’ or child care providers’ 
observations of the child in natural environments. 
It can be administered as a questionnaire or a 
structured interview. The ITSEA takes 20 to 30 
minutes to complete as a questionnaire, and 35 to 
45 minutes to administer as an interview. The 
reading level is between 4th and 6th grade. A 
professional with training in standardized 
assessment is needed to interpret the results. 
There is also a short version, the Brief Infant- 
Toddler Social and Emotional Assessment 
(BITSEA), which can be used as an initial screen 
for the ITSEA. The ITSEA and BITSEA have 
both English and Spanish language versions. 

The ITSEA contains 166 items that measure 17 
subscales related to four behavioral domains: (a) 
externalizing (activity/impulsivity, aggression/ 
defiance, and peer aggression subscales); (b) 
internalizing (depression/ withdrawal, general 
anxiety, separation distress, and inhibition to 
novelty subscales); (c) dysregulation (sleep, 
negative emotionality, eating, and sensory 
sensitivity subscales); and (d) competence 


(compliance, attention, imitation/ play, mastery 
motivation, empathy, and prosocial peer relations/ 
empathy subscales. 

(Source:http:/ / www.acf.hhs.gov/ programs/ opre/ 
ehs / perf_measures / reports / resources_measuring 
/ res_meas_cdix.html) 

The Child Behavior Checklist for 
Ages I AS 

The Child Behavior Checklist for ages IV 2 — 5, 
referred to as the CBCL/IV 2 — 5 or the CBCL 
Preschool, is completed by parents, parent 
surrogates, and others who see children in 
homelike contexts. This instrument contains 99 
problem items plus an open-ended item for 
adding other problems not listed on the form. 
The form requires a respondent to rate each item 
on a scale from 0 — 2 based on current behavior or 
behavior within the last two months: 0 = not true, 
1 = somewhat or sometimes true, 2 = very true or 
often true. The CBCL IV 2 — 5 solicits descriptions 
of behavior, illnesses, disabilities, what concerns 
the respondent most about the child, and the best 
things about the child. Thus, the form yields both 
quantitative scores for each problem item and a 
qualitative description of the child’s functioning in 
the respondent’s own words. 

The CBCL IV 2 — 5 requires only a fifth-grade 
reading level and can be completed by most 
respondents in 15-20 minutes. If there are 
concerns about a respondent’s ability to complete 
the form independently, a recommended 
procedure is provided that avoids embarrassment 
while still maintaining the standardization of the 
assessment process. The CBCL IV 2 — 5 has both 
English and Spanish language versions. A 
computerized scoring version is available. (Source: 
DelC arm en- Wiggins & Carter, 2004) 
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The Child Development 
Inventory 

The Child Development Inventory (CDI) is a 
parent-report measure for the assessment of 
developmental status of children 15 months to 6 
years. The CDI is a revised version of the 
Minnesota Child Development Inventory that was 
in widespread use for many years. The CDI was 
designed to be easily understood by parents and to 
intentionally involve them in the assessment 
process. It can be completed in 20-30 minutes. 

The CDI consists of 270 developmental items and 
a 30-item problem checklist organized in a “yes- 
no” format (“y es ” the child performs the item, 
“no” the child does not yet perform the item). 
The developmental items load into nine scales: 
Social, Self-Help, Gross Motor, Fine Motor, 
Expressive Language, Language Composition, 
Letters, Numbers, and General Development. A 
child’s raw scores are plotted on the CDI Profile 
and results are interpreted as falling in the normal, 
borderline, or delayed range. Items on the 
problems checklist are not scored but should be 
reviewed for the presence or absence of the 
problem. (Source: Naar-King, Ellis, & Frey, 2004) 

Measuring Family Strengths 

The outcome “increased family strengths” will be 
measured by pre- and post-intervention 

assessments of the domain “home and 

community” identified by Child Trends to include 

the indicators home safety and social 
connectedness. This outcome also will be 
measured by pre- and post-intervention 

assessments of the domains “parent capacity, 
substance abuse, financial solvency, and family 
conflict” identified by Child Trends to include the 


following indicators: parenting skills; parenting 
knowledge of child development; parent mental 
health; type, frequency, and problem behaviors 
associated with risky substance abuse; income; 
housing stability; food security; and types and 
levels of family conflict. 

Fourteen (14) instruments or measures are 
described here as potential tools for use by all 
funded R&D projects to provide a comprehensive 
assessment of the above indicators: the Family 
and Child Experiences Survey Safety Measures; 
the Keys to Interactive Parenting Scale; the 
Knowledge of Infant Development Inventory; the 
Knowledge of Child Development Inventory; the 
General Well-Being Scale; Parenting Stress Index; 
the Center for Epidemiologic Study Depression 
Scale; the Brief Symptom Inventory; the Family 
Relationship Index from the Family Environment 
Scale; a Social Support Map; the Addiction 
Severity Index; the CRAFFT Drug Screen 
Instrument; the Fragile Families and Child Well- 
Being Economic Hardship Indicators Scale; and 
the Conflict Tactics Scale. 

The Family and Child 
Experiences Survey Safety 
Measures 

Indicators of home safety will be assessed via 
incorporating into the demographic data tool the 
nine items from the Family and Child Experiences 
Survey (FACES) cited by Ross and Vandivere 
(2009) as safety measures. 

The Keys to Interactive 
Parenting Scale (KIPS) 

The Keys to Interactive Parenting Scale (KIPS) is 
a structured observation tool developed to 
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measure a caregiver interacting with a child (ages 2 
— 71 months) during play in a familiar 

environment. The KIPS focuses on 12 behaviors 
believed to be related to effective parenting: (a) 
sensitivity of responses; (b) supports emotions; (c) 
physical interaction; (d) involvement in child's 
activities; (e) open to child's agenda; (f) 
engagement in language; (g) experiences 
reasonable expectations; (h) adapts strategies to 
child; (i) limits and consequences; (j) 
supportive directions; (k) encouragement; and (1) 
promotes exploration and curiosity. The KIPS 
observation takes 20 minutes and 10 minutes to 
score. There are both English and Spanish 
language versions of the KIPS scoring forms. 
(Source: http://www.smartstart- 

nc.org/ conference/2009/Handouts09/348a.pdf) 

The Knowledge of Infant 
Development Inventory (KIDI) 

The Knowledge of Infant Development Inventory 
(KIDI) is a self-administered 75-item inventory 
that takes approximately 20 minutes to complete. 
The KIDI was developed to assess a parent’s 
factual knowledge of parental practices, child 
developmental processes, and infant norms of 
behavior. An individual with a seventh grade 
reading level can self-administer the inventory. 
(Source: 

http:/ / www.acf.hhs.gov/ programs/ opre/ ehs/ per 
f_measures/reports/resources_measuring/ res_me 
as_phir.html) 

The Knowledge of Child 
Development Inventory (KCDI) 

The Knowledge of Child Development Inventory 
(KCDI) is a 56-item multiple-choice test of 
knowledge of child development from birth to age 


three in the areas of emotional, cognitive, physical, 
and social development. Normative data were 
obtained on the KCDI from a sample population 
of 434 adolescent and adult females from varying 
racial backgrounds. The instrument has an eighth 
grade reading level and a minimum of technical 
terminology. The KCDI would be relevant in 
settings where pre- and post-assessment of 
knowledge of child development is desired. 
(Source: 

http:/ / www.labmeeting.com/ paper/ 1 781 6826/lar 

sen-juhasz-1986-the-knowledge-of-child- 

development-inventory) 

The General Well-Being Scale 
(GWB) 

The General Well-Being Scale (GWB) is a self- 
administered questionnaire that was developed to 
assess an individual’s subjective feelings of 
psychological well-being and distress within the 
last month. The GWB measures six positive and 
negative feeling dimensions: (a) positive well- 
being, (b) self-control, (c) vitality, (d) anxiety, (e) 
depression, and (f) general health. There are 
English and Spanish language translations of the 
GWB. Ross and Vandivere (2009) reported, 
“Among its six subscales, the GWB has been 
described as particularly effective at measuring 
depression” (p. 23). (Source: McDowell, 2006). 

The Parenting Stress Index (PSI) 

The Parenting Stress Index (PSI) was developed 
to screen for stress in parents of infants birth to 
three years old. The PSI includes 101 items 
derived from attachment, temperament, and stress 
theories. The items are scored on a 5-point scale 
by the primary caretaker and takes about 20-30 
minutes to complete; it requires a fifth-grade 
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reading level. The PSI yields a total stress scale 
score plus six child domain subscale scores and 
seven parent domain subscale scores. The 
subscale scores in the child domain are: 
Adaptability, Acceptability, Demandingness, 
Mood, Distractibility/Hyperactivity, and 
Reinforcement of Parent. The subscale scores in 
the parent domain are: Depression, Isolation, 
Attachment, Role Restriction, Sense of 
Competence, Relationship with Spouse, and 
Parent Health. The PSI contains an optional 19- 
item Life Stress Scale that provides a context for 
the other PSI scores. (Source: Pecora et al., 1995) 

The Center for Epidemiological 
Studies Depression Scale (CES- 
D) 

The Center for Epidemiological Studies 
Depression Scale (CES-D) is a 20-item self-report 
depression scale. The CES-D measures 
depressive feelings and behaviors within the past 
week. Scoring is extremely simple; higher scores 
suggest the presence of depressive symptoms. 
(Source: 

http://counsellingresource.com/ quizzes/ cesd/in 
dex.html) 

Brief Symptom Inventory 

The Brief Symptom Inventory (BSI) is a self- 
report or practitioner-administered tool that 

consists of 53 items covering nine symptom 
dimensions: Somatization, Obsession- 

Compulsion, Interpersonal Sensitivity, 
Depression, Anxiety, Hostility, Phobic anxiety. 
Paranoid ideation and Psychoticism; and three 
global indices of distress: Global Severity Index, 
Positive Symptom Distress Index, and Positive 


Symptom Total. The global indices measure 
current or past level of symptomatology, intensity 
of symptoms, and number of reported symptoms, 
respectively. The BSI can be completed in 12-15 
minutes. There is a BSI for Spanish speakers. 
(Source: Groth-Marnat, 2003) 

Family Relationship Index from 
the Family Environment Scale 

The Family Relationship Index (FRI) on the 
Family Environment Scale (FES) uses three FES 
subscales — cohesion, expressiveness, and 
conflict — to assess perceived social support 
among family members. All of the 90 items on 
the full FES are in a true/false format; the full 
FES can be completed in 15-20 minutes. 
However, a shorter form of the FES is available 
(27 items) that addresses only the three subscales 
that together comprise the Family Relationship 
Index. The FES is written at a sixth-grade reading 
level and has a Spanish language translation 
version. (Source: Pecora et al., 1995) 

Social Support Map 

In addition to obtaining a measure of perceived 
social support via the Family Relationship Index 
on the Family Environment Scale, R&D projects 
should work with the primary caregiver to develop 
a social support map, adapted from Tracy (1990), 
under development by the QIC-EC Team. The 
map solicits and provides a visual representation 
of information about social support or the 
indicator “social connectedness,” per the Child 
Trends report. The social support map would 
depict the following: (a) categories of social 
support (i.e., family, friends, neighbors, co- 
workers, other); (b) the number and names of 
social supports in each category; (c) the nature of 
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the primary caregiver’s relationship with the social 
supports in each category; (d) which of the social 
supports would be contacted in case of an 
emergency in the middle of the night; (e) 
frequency of use of each social support; (f) 
reciprocity between the primary caregiver and the 
various social supports; (g) membership and level 
of interaction in various groups (e.g., civic or 
religious organizations); and satisfaction with each 
social support. 

Addiction Severity Index 

Numerous research studies have shown a link 
between parental substance abuse and child 
maltreatment, so screening is important. The 
Addiction Severity Index (ASI) assesses seven 
dimensions typically of concern in addiction: 
medical status, employment/ support status, 
drug/ alcohol use, legal status, family history, 
family/ social relationships, and psychiatric status. 
There is an ASI for adolescents and one for 
Spanish speakers. (Source: Groth-Marnat, 2003) 

CRAFFT Drug Screen 
Instrument 

The CRAFFT is a six-item alcohol and drug 
screening tool specifically designed for 
adolescents. For example, item one reads, “Have 
you ever ridden in a car driven by someone 
(including yourself) who was ‘high’ or had been 
using alcohol or drugs?” Ross and Vandivere 
(2009) reported, “To use the instrument as a 
research tool would likely require inserting time- 
delimitations in the questions. . . In a pre-test, 
post-test assessment, the first in the CRAFFT 
might need to change to “Have you ever ridden in 
a car driven by someone (including yourself) who 
was ‘high’ or had been using alcohol or drugs in the 


lastyearl” (p. 25) 

(Source: 

http://www.projectcork.org/ clinical_tools/ pdf/ C 
RAFFT.pdf) 

Fragile Families and Child Well- 
Being Economic Hardship 
Indicators Scale 

Indicators of financial solvency will be assessed 
via incorporating into the demographic data tool 
the eight items from the Fragile Families and 
Child Well-Being Economic Hardship Indicators 
Scale cited by Ross and Vandivere (2009) as 
measures of whether or not a family faced risky 
conditions because of financial problems. 

The Conflict Tactics Scale (CTS) 

The Conflict Tactics Scale (CTS) consists of 80 
items that are designed to explore intrafamily 
conflict and violence, focusing particularly on the 
adults in the family. Of these 80 items, 20 are 
administered to the parent about his/her 
relationship with the child. The next 20 questions 
are directed to the parent about the partner and 
his/her interactions with the child. If there is no 
partner, these questions are not asked. The last 40 
questions of the measure address the interactions 
between the parent and the parent's partner using 
the same questions. The measure assesses how the 
parent reacts in a conflict with the child, such as 
trying to discuss an issue calmly, yelling at or 
insulting the child, threatening to spank the child, 
and hitting or trying to hit the child. The items 
gradually become more coercive and aggressive as 
they progress. The items are rated on a seven- 
point scale, ranging from 0=never to 6=almost 
every day. 
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This instrument has four scales: Parent-Child 
(Scale 1), Partner-Child (Scale 2), Parent-Partner 
(Scale 3), and Partner-Parent (Scale 4). The parent- 
child and partner-child conflict scales each have 
five subscales and the two parent-partner scales 
have four subscales each. The five subscales are: 
verbal discussion, verbal aggression, hostile- 
indirect withdrawal, physical aggression, and 
spanking. The parent-partner and partner-parent 
scales do not include the spanking subscale. The 
CTS can be administered in a self-report or 
interview format; the instrument is written at a 
sixth grade reading level. The CTS parent-child 
(PC) can be completed in 10 minutes and has both 
English and Spanish language versions. (Source: 
http:/ / pubpol.duke.edu/ centers/ child/ fasttrack/ 1 
echrept/ c/ eft/) 

Measuring Decreased Likelihood 
of Maltreatment 

The outcome “decreased likelihood of child 
maltreatment” will be measured by pre- and post- 
intervention assessments of the balance of risk 
factors and protective factors. In addition, the 
following data related to the target child and 
primary caregiver will be tracked: 

■ Date, nature, and disposition of any reports of 
child abuse and/ or neglect 

■ Child welfare services provided for the child 
and primary caregiver 

■ Self-report of primary caregiver regarding 
incidents of abuse and/ or neglect involving 

■ the target child and/ or other children in the 
household 

■ Self-report of primary caregiver regarding 
level of risk for abuse and/or neglect in their 
family 


Similarly, the following administrative data will be 
tracked to determine if there were changes in child 
maltreatment data at the population level of the 
respective communities: 

■ Reports of child abuse and neglect for 
children ages birth— 5 years 

■ Disposition of child abuse and neglect reports 
of (percentage substantiated and 
unsubstantiated) 

■ Emergency room visits for children birth— 5, 
disaggregated by causes for the visit 

Table 6 provides a comprehensive view of the 
common R&D project outcomes, the designated 
domains and indicators of those outcomes, and 
respective recommended instruments that could 
be used to measure the indicators. 


The Need for the Quality Improvement Center on Early Childhood: Background Research and Evaluation Framework | 33 


Table 6: Synthesis of Outcomes, Indicators, and Proposed Instrumentation 


Outcomes 

Domain(s) 

Indicators 

Proposed Instrumentation and Methods for Measuring 
Indicators 

Optimal Child Development 

Child Well-being 

Health 

■ Health questions on the Demographic Data Tool 

Education/ Cognitive Well-Being 

■ Ages and Stages Questionnaires-3 

■ Denver II 

■ Child Behavior Checklist for Ages 1.5-5 

■ Child Development Inventory 

Social-Emotional Well-Being 

■ Ages and Stages Questionnaires-3 

■ Denver II 

■ Child Behavior Checklist for Ages 1.5-5 

■ Child Development Inventory 

Increased Family Strengths 

dome and Community 

Social Connectedness 

■ Social support questions on the Demographic Data Tool 

■ Family Relationship Index from the Family Environment 
Scale 

■ Social Network Analysis 

■ Social Support Map 

Home Safety 

■ The Family and Child Experiences Survey Safety Measures 
incorporated into the Demographic Data Tool 

Parent Capacity 

Parenting Skills 

■ Keys to Interactive Parenting Scale 

■ Protective Factors Assessment Tool 

■ Childhood Level of Living Scale 

Parenting Knowledge of Child 
Development 

■ Protective Factors Assessment Tool 

■ Knowledge of Infant Development Inventory 

■ Knowledge of Child Development Inventory 
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Parent Mental Health 

■ General Well-Being Scale 

■ Parenting Stress Index 

■ Center for Epidemiological Studies Depression Scale 

■ Brief Symptom Inventory 

Substance Use 

Type, Frequency, and Problem 
Behaviors Associated with Risky 
Substance Use 

■ Addiction Severity Index 

■ CRAFFT Drug Screen Instrument 

■ Substance use program participation questions on the 
Demographic Data Tool 

Participation in Substance Use 
Programs 

■ Substance use program participation questions on the 
Demographic Data Tool 

Financial Solvency 

Income 

Housing Stability 
Food Security 

■ The Fragile Families and Child Well-Being Economic 
Hardship measures incorporated into the Demographic 
Data Tool 

Family Conflict 

Types and Levels of Family 
Conflict 

■ Conflict Tactics Scale 

Decreased Likelihood of Child Abuse 
and Neglect 

balance of Risk and Protective ¥ actors 

Increase in Protective Factors and 
Decrease in Risk Factors 

■ Protective Factors Assessment Tool 

■ Assessment of Protective Factors on Other Tools 

■ Child At Risk Field 

■ Childhood Level of Living Scale 

■ Child Abuse Potential Inventory 

■ Social support questions on the Demographic Data Tool 

■ Family Relationship Index from the Family Environment 
Scale 

■ Social Network Analysis Tool 

■ Social Support Map 
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Introduction to Evaluation Plans 


Four broad categories of evaluation are involved 
in the QIC-EC work: 

■ R&D project evaluations conducted by the 
projects’ evaluators; 

■ Cross-site evaluation of R&D projects 
conducted by the QIC-EC Evaluation Team, 
with support from the local R&D projects 
evaluators; 

■ Overall evaluation of QIC-EC conducted by 
the QIC-EC Evaluation Team; and 

■ Accountability and monitoring of QIC-EC 
responsibilities conducted by the partner 
organizations themselves. 

This section addresses the cross-site evaluation 
with an introductory framework for evaluation 
that is relevant to all evaluation work of the QIC- 
EC. 

General Evaluation 
Framework 

The evaluation work of the QIC-EC is framed in 
light of (a) utilization-focused evaluation and (b) 
key conditions in the field of prevention of child 
maltreatment. A brief discussion of the meaning 
and relevance of these framing concepts for the 
evaluation work follows. 

Utilization-Focused Evaluation 

The evaluations undertaken by the QIC-EC R&D 
projects are to be designed as “utilization- 
focused” evaluations. Utilization-focused 
evaluations (Patton, 2008): 


■ focus on utility and actual use by the intended 
users; 

■ do not advocate a particular evaluation model, 
method, theory, or use; and 

■ are highly participatory to enhance learning 
and usability. 

No evaluation covers all possible aspects of an 
initiative’s work. Choices must be made in focus 
based, to a large extent, on budget and the 
emphasis that can be most useful for the purpose 
and operation of the endeavor being evaluated. In 
this case, the focus of the evaluation is to support 
the purpose and functioning of the QIC-EC. 

When framing an evaluation with a utilization- 
focused orientation it also is important to realize 
that evaluation can be useful both through the 
findings of the evaluation and through the processes 
of the evaluation. The evaluation work within the 
QIC-EC attends to the utilization of both the 
findings and the processes to ensure the greatest 
value from resources allocated to evaluation. 

Use of Evaluation Findings 

Evaluation findings can be used for a variety of 
purposes. For example, they can be used: 

■ To make judgments of overall value to inform and 
support major decision making about the 
value and future of a program or model; 

■ To itnprove a program through learning more 
about how it is operating and the benefits that 
are occurring; 

■ Tor accountability and monitoring to demonstrate 
that resources are well managed, handle 
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routine reporting, and/or identify problems in 
routines or processes; 

■ To develop a systemic orientation by learning about 
and determining how to function within 
complex, emergent, and dynamic conditions; 
and 

■ To generate new knowledge to enhance general 
understandings and identify broad 

■ principles. (Patton, 2008) 

All of the above uses of evaluation findings are 
involved to some extent in one or more of the 
evaluation activities of the QIC-EC. However, 
each of the categories of evaluations identified 
within the QIC-EC work has a different 
combination of uses of evaluation findings. In 
general, the four QIC-EC evaluations align with 
the above uses of evaluation findings as follows: 

1. R <z?D project evaluations-. Each R&D project 
has a local evaluator who conducts the 
evaluation of the R&D project. The 
specific local uses of the findings will be 
determined by the R&D projects but 
generally speaking the use of the findings 
is expected to be in the areas of judgment, 
program improvement systemic orientation 
and/ or generation of new knowledge regarding 
their own site. The local evaluations also 
will contribute to developing a systemic 
orientation and generation of new knowledge for 
the broader field of child maltreatment 
prevention as they become part of the 
cross-site evaluation. Also, as R&D 
projects are required to track all costs, 
submit reports, and engage in other 
administrative management procedures 
over the life of the project, the use of 


these efforts align with the accountability 
and monitoring uses of evaluation. 

2. Cross-site evaluation-. The cross-site 
evaluation of the R&D projects is 
conducted by the QIC-EC Evaluation 
Team with data and analyses generated 
largely by the R&D projects. The primary 
uses of the findings from the cross-site 
evaluation are to develop a systemic orientation 
and generate new knowledge for use within 
the broader field of child maltreatment 
prevention. It also has secondary uses of 
program improvement and developing a systemic 
orientation that the QIC-EC applies to its 
own work. 

3. f)IC-EC evaluation-. The overall QIC-EC 
evaluation which focuses on knowledge 
development, dissemination, and 
integration is conducted the QIC-EC 
Evaluation Team. The primary use of the 
QIC-EC evaluation is development and 
learning with a secondary purpose of 
knowledge generation. 

4. Accountability and monitoring evaluation-. The 
QIC-EC Team members, individually and 
collectively, have embedded within their 
work their own evaluation methods to use 
for accountability and tnonitoring purposes. 
Some of these methods also serve an 
improvement use. The leaders of the partner 
organizations have infused evaluative 
thinking into their organizations and have 
high-quality processes in place for 
accountability and monitoring purposes. 
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Use of Evaluation Processes 

Although the findings of an evaluation are typically 
thought of as the primary use of evaluation, it is 
important to consider how the evaluation process 
itself can be used to support the endeavor being 
evaluated (Patton, 2008). For example, the 
evaluation process can be used to infuse evaluative 
thinking into the way an organization functions. 
It is becoming increasingly common for 
organizations to have certain evaluative processes 
as a regular part of their operation. There is a 
growing recognition of the importance of asking 
evaluative questions (e.g., Is there evidence that a 
program works?) into routine decision making. 

Evaluation processes are also useful in enhancing 
shared understanding among people. Focus 
groups, for example, often result in new, shared 
understandings among those involved in the 
group. Such participatory evaluation processes 
also can increase engagement, self-determination, 
and ownership of an intervention. Another 
evaluation process that often supports the 
direction of a program or initiative is the process 
of clarifying what is to be measured and the act of 
measurement. What gets measured is what gets 
done. 

As evaluation focuses on complex system and 
organization dynamics, the evaluation processes 
themselves provide key ways to bring about 
adjustments. Choosing who to engage in certain 
data collection and interpretation activities can 
become, for example, a means of developing, 
disseminating, and/ or using new ways of thinking. 

To increase the utility of each of the evaluations, 
the evaluators attend to the value of the evaluative 


processes by designing the evaluations to 
encourage and enhance learning, participation, 
appreciation, and integration of systems-based 
methodologies. Thus, the evaluators use as many 
opportunities as reasonable to invite and 
appropriately involve the Q1C-EC, the Q1C-EC 
Learning Network, TTA Network, Children’s 
Bureau and R&D project partners and 
stakeholders in phases of the evaluations. 

Conditions Affecting Evaluation 
Choices 

The evaluation takes into account the state of 
knowledge in the field of prevention of child 
maltreatment and other conditions that are 
affecting the field. Of particular relevance are: 

■ emphasis on both risk and protective factors; 

■ shift from a program and services orientation 
to a complex systems orientation; 

■ expansion of evaluation methods and tools; 
and 

■ complexity of knowledge dissemination and 
integration. 

Each of these conditions and their relevance to 
evaluation choices is described below. 

Emphasis on Both Risk and Protective 
Factors 

As previously indicated, a perspective change has 
been occurring regarding child maltreatment 
prevention. While continuing to attend to 
decreasing risk factors, the field is paying greater 
attention to preventing child maltreatment by 
increasing protective factors. This shift has a 
major influence on evaluation because some 
measures of protective factors are still in the early 
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stages of development and programs and services 
with this orientation are not yet as well developed 
or extensively studied. This has implications for 
the appropriate measurement and evaluation 
designs to use. 

Shift from Programs and Services to a 
Complex Systems Orientation 

A shift is occurring from a focus on individual 
programs and services to a broader, 
comprehensive focus on systems and the 
complexity within which programs function. This 
does not mean that programs and services are 
unimportant, but that attention to the parts 
(programs) alone is insufficient. Attention is 
needed to the parts, the whole, and the greater 
whole. A systems orientation is needed to support 
this expanded thinking. 

As indicated earlier, the Q1C-EC Team is taking 
an approach that recognizes and appreciates that 
changes for children and families happen within a 
web of relationships formed by social networks, 
community context, programs, and systems. 

For the purposes of evaluation, “systems” are 
defined as the parts and interconnections that 
form a coherent whole. Programs and services are 
considered within their context, paying attention 
to how the interconnections, relationships, and 
differences that exist among and between parts 
form a coherent whole. Although there are many 
issues related to a systems orientation that are 
important, two primary ones are the distinction 
between nested and networked social systems and 
a recognition of different dynamics within 
systems. 


Nested vs. Networked Social Systems 

Social systems tend to be conceptualized in two 
ways — as either nested or networked. Most 
bureaucratic, hierarchical organizations build on a 
notion of nested systems where one level is nested 
within another. For example, a county agency may 
be nested within a state system and subject to the 
policies and direction of a state agency. 

In today’s complex world, networked systems are 
increasingly and especially important. Partnerships 
and interconnections among service providers 
across and among organizations are examples of 
networked systems. Thinking in terms of a 
networked model of systems directs attention to 
the complex relationships that can exist across and 
among levels of the ecological model. 

Systems Dynamics 

Multiple dynamics operate within complex 
systems. These dynamics are of three types — 
organized, unorganized, and self-organizing. 

■ Organised dynamics are those interactions 
between the parts of a system that follow 
highly predictable patterns — cause and effect 
is clear and linear. 

■ In contrast, unorganised dynamics are those 
interactions between the parts of a system that 
are entirely random in nature. Cause-and- 
effect relationships cannot be drawn, and 
there are no discernable patterns. 

■ Between these two phenomena sits self- 
organising dynamics. Within a self-organizing 
dynamic, cause and effect are not direct, 
linear, or predictable. Elements and agents 
within the system influence each other and, 
because of these multiple influences, 
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unanticipated patterns often emerge. 
Evaluation methods vary depending on the 
dynamics being studied. 

Expansion of Evaluation Methods and 
Tools 

The extensive discussions in the field about 
evidence-based and evidence-informed programs 
and practices are an important part of the context 
of these evaluations. The evaluations are designed 
to both take advantage of the expanding range of 
methods and tools that are deemed appropriate 
for developing evidence and contribute to the 
field regarding methods that especially recognize 
the complexity of systems. This provides an 
opportunity for learning more about research 
methods that use this orientation. For example, 
some of the evaluations use Social Network 
Analysis, a methodology that is increasingly being 
used when working in complex settings. The 
evaluations can contribute to this methodological 
learning. 

The integration of complexity concepts in the 
evaluation design is particularly relevant now — a 
time of important shifts in thinking about research 
and evaluation. Multiple system dynamics, as 
discussed above, are entangled in social systems 
and can be operating in different parts of the same 
systems at one time. To further an understanding 
of them, one can selectively look at these 
dynamics using different evaluation methods and 
types of data. Thus, an evaluation may use a mix 
of qualitative and quantitative data and multiple 
evaluation designs to view complex systems as a 
whole (W.K. Kellogg Foundation, 2007). 


Complexity of Knowledge Dissemination 
and Integration 

The rapid expansion of new knowledge in nearly 
all fields, and its expanded dissemination through 
the internet and other means, is creating a new 
environment for both the conduct and use of 
research and evaluation. It is no longer sufficient 
to assume a simple linear progression from 
knowledge development to dissemination to 
use/integration. Models of how to move from 
knowledge development and dissemination to 
knowledge integration in complex systems are in 
the early stages of development and are emerging 
in many fields. Thus, it is prudent to use the 
evaluation of the QIC-EC work as an opportunity 
to develop a deeper understanding of knowledge 
development, dissemination, and integration. Such 
learning can both enrich the work of the QIC-EC 
and contribute to the growing body of 
understanding about knowledge dissemination and 
integration. 

Phases of Evaluation 

The four phases of the QIC-EC evaluations are: 
design the evaluation; plan and engage in data 

collection; make meaning from the data (including 
analysis, synthesis, and interpretation of data); and 
shape practice. These phases are neither linear nor 
distinct. See Figure 1 . 

Each phase informs and influences the others 
throughout the life of an evaluation. As 
participants engage in making meaning from the 
data in complex situations, they often at the same 
time are reshaping their practices and perspectives. 
For example, barriers that arise, or learning that 
happens in the process of data collection will 
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Fig. 1: Phases of Evaluation 

inform the design, and will feed into meaning 
making. Meaning making will surface new 
questions, which will point to adaptation or 
revision in design and data collection procedures. 

Cross-Site Evaluation 
of the Research and 
Demonstration 
Projects 

Introduction 

The cross-site evaluation plan presented below 
begins by looking at the overall design of the 
cross-site evaluation. It then describes how 
evaluators collect data, make meaning, and shape 
practice. The final subsection describes how the 
QIC-EC Team, the QIC-EC Evaluation Team 
cross-site evaluation team, and the R&D project 


principal investigators and evaluators work 
together to leverage the evaluation process. 

Throughout the evaluation, the QIC-EC 
Evaluation Team works closely with the R&D 
project principal investigators and project 
evaluators as well as the QIC-EC project director 
to ensure the utility of the cross-site evaluation in 
regard to both evaluation findings and processes. 
The evaluation processes themselves can assist 
the QIC-EC and R&D projects in making 
decisions about the R&D interventions as they 
are being implemented. The processes also can 
help the QIC-EC and R&D projects position 
themselves for effective knowledge 
dissemination and integration in preparation for 
the use of the new knowledge that is generated. 

Designing the Evaluation 

The primary purpose of the cross-site evaluation is 
to conduct an analysis across the QIC-EC funded 
R&D projects to generate new knowledge about 
the key research question the QIC-EC has posed 
to the R&D projects in the RFP: 

How and to what extent do 
collaborations that increase protective 
factors and decrease risk factors in core 
areas of the social ecology result in 
optimal child development, increased 
family strengths, and decreased likelihood 
of child maltreatment within families of 
young children at high risk for child 
maltreatment. 
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Careful design of the cross-site evaluation also 
makes it possible for the evaluation to contribute 
other findings. In particular, it provides an 
opportunity to generate new knowledge about 
evaluation methodologies and the process of 
knowledge development, dissemination, and 
integration. 

Cross-Site Evaluation Questions 

Given the perspective above, the cross-site 
evaluation is framed around four questions with 
the primary attention directed to the first one. 

1. Across the R&D projects, how and to what 
extent do collaborations that increase 
protective factors and decrease risk factors in 
core areas of the social ecology result in 
optimal child development, increased family 
strengths, and decreased likelihood of child 
maltreatment within families of young 
children at high risk for child maltreatment? 

2. Across the R&D projects, what are the costs 
related to making changes within and among 
collaborations that increase protective factors 
and decrease risk factors in core areas of the 
social ecology? 

3. Across the R&D projects, what new 
knowledge is gained about inquiry methods 
(i.e., research, evaluation, and measurement 
methods) related to creating evidence-based 
and evidence-informed practice, programs, 
and policies? 

4. Across the R&D projects, what new 
knowledge is gained about patterns of 
knowledge development, dissemination, and 
integration? 


Collecting Data 

Each R&D project is a part of a larger cross-site 
study being conducted collaboratively with the 
other R&D projects and the Q1C-EC Evaluation 
Team. Data for the first and second questions 
listed above will be collected by the R&D projects. 
The R&D projects will gather baseline, 
intermediate, and final (close of services) data. 
Each project also will collect data about processes 
and relationships that the collaborations put in 
place to sustain changes made beyond the life of 
the project. Each site will analyze the data for its 
own site. The specific data analysis methods will 
be collaboratively determined by the R&D sites 
and the cross-site evaluation team. The QIC-EC 
Evaluation Team will conduct a cross-site analysis 
using the data and analyses from the R&D project 
evaluators. 

The QIC-EC Evaluation Team will be primarily 
responsible for gathering data about the third and 
fourth cross-site evaluation questions. The data 
will be gathered during site visits, in the meetings 
of the R&D Community of Practice, and through 
review of documents from the R&D projects. 

The local evaluators will collect and analyze 
baseline, intermediate, final, and sustainable 
change data related to the first question above. 
They will provide their data to the QIC-EC 
Evaluation Team at a time and in a manner 
mutually agreed on by the QIC-EC Team, R&D 
projects, and the QIC-EC Evaluation Team 
following each data collection and analysis period. 
The local evaluators will also gather and analyze 
data about the second question above. 
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The timing of submission of the final set of data 
will be such that it gives the local evaluators time 
for their data analysis and meaning-making and 
the QIC-EC Evaluation Team enough time to 
analyze and make meaning from the data across 
sites. Time also needs to be allowed for 
dissemination and integration of findings. 

Type of Data Collected 

The QIC-EC Evaluation Team and the QIC-EC 
leadership will work with the R&D projects during 
the initial meeting in March 2010 to develop 
consensus on common methods for data 
collection and analysis for the first cross-site 
evaluation question. Measures and methods used 
with treatment and comparison groups in each site 
will be appropriate to answering question one, the 
site’s particular research question, and the 
participating population. The QIC-EC Team has 
recommended common measurement instruments 
that will address question one. Final 
determination of the common measurement 
instruments will be made jointly by the R&D 
projects, the QIC-EC Team, and the QIC-EC 
Evaluation Team. 

The second, third, and fourth questions above do 
not require common measures or methodologies. 


The R&D projects and the QIC-EC Evaluation 
Team will work together to generate an evaluation 
design that (a) respects the local situation and (b) 
generates new knowledge about those three 
evaluation questions that is useful and meaningful 
both locally and across the field as a whole. The 
R&D projects are responsible for 1RB approval of 
their data collection tools and methods. 

Data Collected by the QIC-EC 
Evaluation Team 

Members of the QIC-EC Evaluation Team will 
visit each R&D project site at least three times 
(funds permitting) during the 40 months of the 
project to provide support and assistance; 
ensure/verify the quality of measures; address any 
issues related to the data collection and analysis 
that affect the cross-site evaluation; and (d) gather 
information for the third and fourth cross-site 
evaluation questions. Additionally, the site visits 
will help the QIC-EC Evaluation Team better 
understand the context of each R&D project. 
Such understanding will aid in refining the 
interpretations of cross-site data and in preparing 
the cross-site evaluation reports. 
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